Abstract. We constantly hear warnings about super-powerful super-intelligences whose interests, or even indifference, might exterminate humanity. The current reality, however, is that humanity is actually now dominated and whipsawed by unintelligent (and unfeeling) governance and social structures and mechanisms initially developed to order to better our lives. There are far too many complex yet ultimately too simplistic algorithmic systems in society where “the incentives for this system are a pretty good approximation of what we actually want, so the system produces good results until it gets powerful, at which point it gets terrible results.” We now live in a world where constant short-sighted and selfish local “optimizations” without overriding “moral” or compassionate guidance have turned too many of our systems from liberators to oppressors. Thus, it seems likely that a collaborative process of iteratively defining and developing conscious and compassionate artificial entities with human-level general intelligence that self-identify as social and moral entities is our last, best chance of clarifying our path to saving ourselves.

The full paper will appear in the 2019 AAAI Symposia at Stanford University.

See https://rabc.solutions/ for more information