Book contents
- Planet in Peril
- Reviews
- Planet in Peril
- Copyright page
- Dedication
- Epigraph
- Contents
- Fictional Vignettes
- Acknowledgements
- 1 Introduction
- Part I Existential Threats: The Four Most Pressing Dangers Facing Humankind
- Part II Strategies and Obstacles: The Solutions We Need, and What’s Preventing Them from Being Realized
- 6 How to Beat Climate Change
- 7 Wise Governance for Nukes and Pandemics: Where to Go Faster and Where to Slow Down
- 8 Controlling Things Versus Controlling Agents: The Challenge of High-Level AI
- 9 The International Dimension: Where Every Solution Stumbles
- Prologue to Parts III, IV, and V
- Part III Sensible Steps for Today’s World: Powerful Measures We Can Implement Right Away
- Part IV The Middle-Term Goal: New International Tools for the Late Twenty-First Century
- Part V The Long-Term Goal: Envisioning a Mature System of Global Governance for the Twenty-Second Century
- Endnotes
- Bibliography
- Index
8 - Controlling Things Versus Controlling Agents: The Challenge of High-Level AI
from Part II - Strategies and Obstacles: The Solutions We Need, and What’s Preventing Them from Being Realized
Published online by Cambridge University Press: 13 October 2022
- Planet in Peril
- Reviews
- Planet in Peril
- Copyright page
- Dedication
- Epigraph
- Contents
- Fictional Vignettes
- Acknowledgements
- 1 Introduction
- Part I Existential Threats: The Four Most Pressing Dangers Facing Humankind
- Part II Strategies and Obstacles: The Solutions We Need, and What’s Preventing Them from Being Realized
- 6 How to Beat Climate Change
- 7 Wise Governance for Nukes and Pandemics: Where to Go Faster and Where to Slow Down
- 8 Controlling Things Versus Controlling Agents: The Challenge of High-Level AI
- 9 The International Dimension: Where Every Solution Stumbles
- Prologue to Parts III, IV, and V
- Part III Sensible Steps for Today’s World: Powerful Measures We Can Implement Right Away
- Part IV The Middle-Term Goal: New International Tools for the Late Twenty-First Century
- Part V The Long-Term Goal: Envisioning a Mature System of Global Governance for the Twenty-Second Century
- Endnotes
- Bibliography
- Index
Summary
Whereas today’s “narrow” AI machines can be easily controlled, the probable advent over the coming decades of machines endowed with Artificial General Intelligence (AGI) will pose a far more difficult control challenge – for such machines will function as agents endowed with many human-like capabilities. The AI expert Stuart Russell has proposed a radical new design for the motivational architecture of AI machines, which he calls “humble AI.” Rather than programming AI machines to pursue specific goals, he argues, it would be much safer to give them a single broad goal, namely, maximizing the realization of human preferences. The key innovation here lies in programming the machines so that they can never be 100 percent certain about what those human preferences are – a gambit that restores ultimate control over the machines’ behavior to their human overseers. Russell argues persuasively that fundamental safety research in the field of AI needs to be pursued as a much higher priority than it currently receives today.
Keywords
- Type
- Chapter
- Information
- Planet in PerilHumanity's Four Greatest Challenges and How We Can Overcome Them, pp. 106 - 114Publisher: Cambridge University PressPrint publication year: 2022