The Post-Singularity World, and Dystopian Predictions
Posted: Tue, November 20, 2012 | By:
by Richard Loosemore
Here are the facts as we know them.
FIRST: everything (EVERYTHING) depends on the motivation of future superintelligent machines. It doesn’t matter what happens with nanotechnology, or the future of corporations or the military ... all these things take second place in a world in which there is real, human-level intelligence. Note carefully that it is not just the existence or intellect of those AIs that is the issue; what matters is specifically the mechanism that determines what they want to do.
SECOND: nobody has ever built anything remotely approaching a full AI. This matters, because one of the main reasons we are far from real AI at the moment is that nobody knows how to set up the control system (the drives or motivations) in such a way as to make an intelligent system behave coherently. Figuring out a control mechanism for a narrow AI system like an interplanetary probe is child’s play compared with real human level AI.
Why is this such a big deal? Partly it is a matter of the mind-boggling abstractness of the drives required in a system that wanders around performing the whole repertoire of behaviors associated with full intelligence. So, if all you’ve got to do is build a control mechanism for a space probe, you can specify its goals as a set of statements about keeping various parameters within appropriate limits. But what do you do with an AI that is, for example, just starting kindergarten? Do you give it a top level goal of “Go and play!”? There seems to be something fundamentally wrong with the idea of inserting abstract goals into the current design for (narrow) AI control systems.
So this means that whenever someone extrapolates from a simple AI control mechanism (like a utility-function maximizer) and assumes that the scaled-up version would work in a future human-level AI, they are doing so with zero evidence of viability. The disconnect is comparable to someone in the pre-atomic era that the control system of a canoe – the rudder – could be scaled up and used as the control system for a future atomic power plant.
THIRD: there are a very small number of people who specifically study the kinds of motivation mechanisms that would have to be used in a human level AI, and some of the early conclusions to come out of that work indicate that, counterintuitively, it may be extremely hard to create malevolent intelligent systems. Basically, if you try to get your AI up to that level of intelligence, the system will become unstable if the drive system includes elements of malevolence, or if it is designed to be blindly obedient to your violent intentions. That means that in practice someone else who tries to build a peaceful, empathic AI will get their system to work while you are still trying to get your evil AI past the screaming tantrum stage of its development.
To be sure, these ideas about the instability of dangerous AIs are in their infancy, but the fact remains that people (like myself) who actually do study the kind of AI motivation mechanism that has any chance of working (which means: not the crude “utility function” mechanisms) have come to the conclusion that dystopian outcomes of a singularity are starting to look ridiculously implausible. We may be wrong, yes, but we are the ones on the leading edge at the moment, developing what looks (to us, at least) like the only coherent proposals for motivation mechanisms, so this conclusion is actually coming from the only game in town right now.
But instead of being replete with exploratory visions of these anti-dystopian futures (to coin a phrase), the meme pool is currently being saturation-bombed by people who tell each other that a machine dystopia is such an obviously likely outcome that nobody could possibly disagree with it.
Sigh! Programmer Error!
Want to leave a Comment? You can, either below, or at Zero State’s Facebook page, HERE
I see that the author of this page (a name I think I recall from SIAI forums) asserts:
“it may be extremely hard to create malevolent intelligent systems. Basically, if you try to get your AI up to that level of intelligence, the system will become unstable if the drive system includes elements of malevolence, or if it is designed to be blindly obedient to your violent intentions.”
And then I see a directive to use Buddhist Right Speech in communications.
I like this. It shows thought. If a created intelligence is of at least human-level in its comprehension, we can assume that it will place high value on knowing what is true so that it can optimize its behavior; it will soon figure out that malevolent strategies are seldom deliberately chosen by the most intelligent humans, and likely come to many similar conclusions. For me the archetypal image is of Leonardo da Vinci, arguably the greatest mind this planet has produced; it’s true he designed many weapons, but in his time this was unavoidable in his field; what fewer people know is that he would often buy wild birds in the market just to set them free.
By Tom Buckner on Nov 20, 2012 at 4:39am