5 - Artificial Intelligence: Extreme Reward and Risk

Summary

Over the coming decades, artificial intelligence and robotics will continue to grow rapidly in power and scope, exerting a transformative impact on our lives. Seven trends are likely: the machines will become progressively more versatile and multifunctional in their capabilities; they will be designed to improve their own software and hardware over time, as they interact with their environment; the line between AI and robots will gradually blur, as machines come to permeate our society at all levels; these machines will be entrusted with complex practical tasks that require them to develop sophisticated commonsense knowledge about the human social world; the machines will be interconnected in functional networks that multiply their powers; such machines will need to be able to refuse to obey certain kinds of human commands, raising fundamental questions about who controls their actions; and the logic of arms races will apply to such machines, impelling nations and corporations to develop increasingly powerful machines as quickly as possible, with only a secondary concern for caution and safety.