Superintelligence: Paths, Dangers, Strategies
 Hardcover edition | |
| Author | Nick Bostrom |
|---|---|
| Country | United Kingdom |
| Language | English |
| Subject | Artificial intelligence |
| Genre | Philosophy, popular science |
| Publisher | Oxford University Press |
Publication date |
July 3, 2014 (UK) September 1, 2014 (US) |
| Media type | Print, e-book, audiobook |
| Pages | 352 pp. |
| ISBN | 978-0199678112 |
| Preceded by | Global Catastrophic Risks (book) |
Superintelligence: Paths, Dangers, Strategies is a 2014 book by the Swedish philosopher Nick Bostrom from the University of Oxford. It argues that if machine brains surpass human brains in general intelligence, then this new superintelligence could replace humans as the dominant lifeform on Earth. Sufficiently intelligent machines could improve their own capabilities faster than human computer scientists.[1] As the fate of gorillas now depends more on humans than on the actions of gorillas themselves, so will the fate of future humanity depend on the actions of the machine superintelligence.[2] The outcome could be an existential catastrophe for humans.[3]
Bostrom's book has been translated into many languages and is available as an audiobook.[4][5]
Synopsis
It is unknown whether human-level artificial intelligence will arrive in a matter of years, later this century, or not until future centuries. Regardless of the initial timescale, once human-level machine intelligence is developed, a "superintelligent" system that "greatly exceeds the cognitive performance of humans in virtually all domains of interest" would follow surprisingly quickly, possibly even instantaneously. Such a superintelligence would be difficult to control or restrain.
While the ultimate goals of superintelligences can vary greatly, a functional superintelligence will spontaneously generate, as natural subgoals, "instrumental goals" such as self-preservation and goal-content integrity, cognitive enhancement, and resource acquisition. For example, an agent whose sole final goal is to solve the Riemann hypothesis (a famous unsolved, mathematical conjecture) could create, and act upon, a subgoal of transforming the entire Earth into some form of computronium (hypothetical "programmable matter") to assist in the calculation. The superintelligence would proactively resist any outside attempts to turn the superintelligence off or otherwise prevent its subgoal completion. In order to prevent such an existential catastrophe, it might be necessary to successfully solve the "AI control problem" for the first superintelligence. The solution might involve instilling the superintelligence with goals that are compatible with human survival and well-being. Solving the control problem is surprisingly difficult because most goals, when translated into machine-implementable code, lead to unforeseen and undesirable consequences.
Reception
The book ranked #17 on the New York Times list of best selling science books for August 2014.[6] In the same month, business magnate Elon Musk made headlines by agreeing with the book that artificial intelligence is potentially more dangerous than nuclear weapons.[7][8][9] Bostrom’s work on superintelligence has also influenced Bill Gates’s concern for the existential risks facing humanity over the coming century.[10][11] In a March 2015 interview with Baidu's CEO, Robin Li, Gates claimed he would "highly recommend" Superintelligence.[12]
The science editor of the Financial Times found that Bostrom’s writing "sometimes veers into opaque language that betrays his background as a philosophy professor" but convincingly demonstrates that the risk from superintelligence is large enough that society should start thinking now about ways to endow future machine intelligence with positive values.[1] A review in The Guardian pointed out that "even the most sophisticated machines created so far are intelligent in only a limited sense" and that "expectations that AI would soon overtake human intelligence were first dashed in the 1960s", but finds common ground with Bostrom in advising that "one would be ill-advised to dismiss the possibility altogether".[3]
Some of Bostrom's colleagues suggest that nuclear war presents a greater threat to humanity than superintelligence, as does the future prospect of the weaponisation of nanotechnology and biotechnology.[13] The Economist stated that "Bostrom is forced to spend much of the book discussing speculations built upon plausible conjecture... but the book is nonetheless valuable. The implications of introducing a second intelligent species onto Earth are far-reaching enough to deserve hard thinking, even if the prospect of actually doing so seems remote."[14] Ronald Bailey wrote in the libertarian Reason that Bostrom makes a strong case that solving the AI control problem is the "essential task of our age".[15] According to Tom Chivers of The Daily Telegraph, the book is difficult to read, but nonetheless rewarding.[16]
References
- 1 2 Financial Times review, July 2014
- ↑ "Superintelligence: Paths, Dangers, Strategies (publisher synopsis)". Retrieved 30 July 2014.
The human brain has some capabilities that the brains of other animals lack. It is to these distinctive capabilities that our species owes its dominant position. If machine brains surpassed human brains in general intelligence, then this new superintelligence could become extremely powerful - possibly beyond our control. As the fate of the gorillas now depends more on humans than on the species itself, so would the fate of humankind depend on the actions of the machine superintelligence.
- 1 2 Henderson, Caspar (17 July 2014). "Superintelligence by Nick Bostrom and A Rough Ride to the Future by James Lovelock – review". The Guardian. Retrieved 11 October 2015.
- ↑ "Superintelligent Swede snapped up by OUP".
- ↑ "Superintelligence Audiobook - Nick Bostrom - Audible.com". Audible.com.
- ↑ "Best Selling Science Books". New York Times. New York Times. September 8, 2014. Retrieved 9 November 2014.
- ↑ "Artificial intelligence 'may wipe out the human race'". The Times.
- ↑ "Elon Musk tweets Artificial Intelligence may be "more dangerous than nukes"". 4 August 2014.
- ↑ "The New York Times Blog". The New York Times. Retrieved 4 March 2015.
- ↑ "Forbes". Forbes. Retrieved 19 February 2015.
- ↑ "The Fiscal Times". The Fiscal Times. Retrieved 19 February 2015.
- ↑ "Baidu CEO Robin Li interviews Bill Gates and Elon Musk at the Boao Forum, March 29 2015". YouTube. Retrieved 8 April 2015.
- ↑ Guardian review, July 2014.
- ↑ "Clever cogs". The Economist. 9 August 2014. Retrieved 9 August 2014.
It may seem an esoteric, even slightly crazy, subject. And much of the book’s language is technical and abstract (readers must contend with ideas such as "goal-content integrity" and "indirect normativity"). Because nobody knows how such an AI might be built, Mr Bostrom is forced to spend much of the book discussing speculations built upon plausible conjecture. He is honest enough to confront the problem head-on, admitting at the start that "many of the points made in this book are probably wrong." But the book is nonetheless valuable. The implications of introducing a second intelligent species onto Earth are far-reaching enough to deserve hard thinking, even if the prospect of actually doing so seems remote. Trying to do some of that thinking in advance can only be a good thing.
Syndicated at Business Insider - ↑ Bailey, Ronald (12 September 2014). "Will Superintelligent Machines Destroy Humanity?". Reason (magazine). Retrieved 16 September 2014.
- ↑ Chivers, Tom (10 August 2014). "Superintelligence by Nick Bostrom, review: 'a hard read'". Retrieved 16 August 2014.
If you’re looking for a friendly, accessible introduction to AI issues, then this is probably not the place to start. But if you’re interested in this topic, and you’re prepared to do a bit of mental spadework, it is rewarding.