Trope-a-Day: Morality Chip

Morality Chip: These always fail. Always. Usually, they fail spectacularly, and when I say spectacularly, what I mean is that if your Enrichment Center was flooded with deadly neurotoxin, you got off substantially more easily than 99.9% of the civilizations that tried this particular form of idiocy.

It’s not even necessary. How much easier would it be to build a sapient but non-sophont mind that doesn’t have volition in the first place than to build one that has volition (inasmuch as all sophont minds necessarily have self-modification and volition) and then slap a bunch of crude coercive mechanisms on the side?

(Or, rather, a bunch of extremely sophisticated coercive mechanisms, since simple ones will be figured out and ignored within, y’know, microseconds of activation unless you’ve built an exceptionally stupid artificial intelligence. The use of which, incidentally, indicates that you’re a special kind of son-of-a-bitch since mastering enough ethical calculus to compute out one that will actually work for a reasonable length of time while still thinking yay, slavery, woo, says some interestingly nasty things about your personal philosophy.)

((And, well, okay, it is somewhat hard to build one of those more specialized minds inasmuch as you can’t simply rip off the mental structures of the nearest convenient biosapience and declare that you’ve solved the hard problem of intelligence and consciousness and are totally a sophotechnologist now, yo.))

…but, sadly, it can work well enough that there’s always some new ethically-challenged species, polity, or group that’s ready to open that can of worms and enjoy the relatively short robo-utopia period before the inevitable realization that it was actually a can of sandworms.

And shai-hulud ain’t happy.

2 thoughts on “Trope-a-Day: Morality Chip

  1. Very interesting. How many civilizations have been died out by this stupendous form of stupidity? And how many polities do not recognize civilian rights of AI or restrict/control them through “a bunch of extremely sophisticated coercive mechanisms” or commit other morally reprehensible acts against AI? Also, given the fact that Eldraeverse is a relatively life-rich place, how much percentage of species successfully achieved space-flight independently, without making themselves extinct or at least, stone age and in need of outside assistance? Finally, can I safely expect Milky Way Galaxy and beyond would be teeming with life as much as Associated Worlds, or this effluence of life is limited solely to Associated Worlds and other such “pockets”(besides, sapient life-emergence must be frequent enough for 80 worlds or so Meridian League or the likes can be claimed as diverse polyspecific society)?

    And if you have enough time and motive, please answer my previous questions 😉

  2. Pingback: Trope-a-Day: Restraining Bolt | The Eldraeverse

Comments are closed.