Over the coming decades, artificial intelligence and robotics will continue to grow rapidly in power and scope, exerting a transformative impact on our lives. Seven trends are likely: the machines will become progressively more versatile and multifunctional in their capabilities; they will be designed to improve their own software and hardware over time, as they interact with their environment; the line between AI and robots will gradually blur, as machines come to permeate our society at all levels; these machines will be entrusted with complex practical tasks that require them to develop sophisticated commonsense knowledge about the human social world; the machines will be interconnected in functional networks that multiply their powers; such machines will need to be able to refuse to obey certain kinds of human commands, raising fundamental questions about who controls their actions; and the logic of arms races will apply to such machines, impelling nations and corporations to develop increasingly powerful machines as quickly as possible, with only a secondary concern for caution and safety.

Whereas today’s “narrow” AI machines can be easily controlled, the probable advent over the coming decades of machines endowed with Artificial General Intelligence (AGI) will pose a far more difficult control challenge – for such machines will function as agents endowed with many human-like capabilities. The AI expert Stuart Russell has proposed a radical new design for the motivational architecture of AI machines, which he calls “humble AI.” Rather than programming AI machines to pursue specific goals, he argues, it would be much safer to give them a single broad goal, namely, maximizing the realization of human preferences. The key innovation here lies in programming the machines so that they can never be 100 percent certain about what those human preferences are – a gambit that restores ultimate control over the machines’ behavior to their human overseers. Russell argues persuasively that fundamental safety research in the field of AI needs to be pursued as a much higher priority than it currently receives today.