If Anyone Builds It, Everyone Dies The Case Against Superintelligent AI

I really appreciate the idea behind this book: raising awareness of the potential dangers of artificial superintelligence (ASI), including the possibility of human extinction. The authors use different scenarios and thought experiments to explain why creating an intelligence more powerful than our own could have catastrophic consequences. I particularly appreciated how they challenged seemingly simple solutions, such as “Why don’t we just build it differently?” Rather than dismissing these arguments, they explored why the problem might be much more complicated than it initially appears. However, while some examples genuinely helped me understand the reasoning, others felt disconnected from the bigger picture. At times, I found myself wondering where a particular scenario was leading and how it would eventually connect to the central argument. The repeated references to the book’s title also started to feel almost like a mantra. I understood the message, but the repetition didn’t necessarily make it more convincing. And one thing that genuinely bothered me was the constant switching between fonts throughout the text. It might seem like a minor detail, but I found it distracting and disruptive to the reading experience. Overall, I think the book raises important questions and presents some interesting arguments, but I wasn’t entirely convinced by the way they were communicated. I appreciated its intention and the opportunity to think more critically about AI development, even if the reading experience itself was sometimes frustrating.

I was really on the fence about if I should read this book. I was already scared about AI before reading it and now I know I severely undererstimated how bad it really is. Do I regret reading the book? Not in the slightest. If you ask me, EVERYONE should read this book. The actual dangers AI poses if it does not get regulated ASAP is something everyone should now about. Everyone has to know about! It's bad, but the more people now about it, the better are our chances to stop AI before it's too late. So if you are on the fence about reading this book, please read it right now!

This book is rightfully recommended by a wide range of credible and prominent people, including nobel price winners (Ben Bernanke), national security experts (Jon Wolfsthal), MIT professors (Max Tegmark) and more. The main message of the book is: "If any company or group, anywhere on the planet, builds an artificial superintelligence using anything remotely like current techniques, based on anything remotely like the present understanding of AI, then everyone, everywhere on Earth, will die." This is relevant for everybody, because most of the major AI companies are currently trying to build such a superintelligent AI, with wildly insufficient precautions. Most people are not aware that there is a real risk that this will kill everyone on Earth. Part 1 of the book explains the very strong arguments why this is the case: why such AIs develop weird goals which were not intended by human developers, how these are practically guaranteed to have motivations incompatible with human life, and how such a superhuman AI will find it very easy to kill literally every human on Earth. Part 2 illustrates these abstract arguments with a detailed example scenario how such an AI can come into existence, how it evades human constraints (even though the fictional AI company is more careful than present-day real AI companies), and how it then kills all humans and takes over as many galaxies as it can. This is not meant as a detailed prediction; as explained in part 1, there are many different ways in which such an AI can end up killing all humans. This part 2 is still helpful for making the general arguments from part 1 more salient. Part 3 is the authors' call to action: how can we prevent such an outcome? What is the current state of AI safety, and what do we urgently need to do to ensure no homicidal superintelligent AI ever gets built? The book is very well written and clearly explains the arguments, accessible to a lay audience, on only 250 pages. Many more details and background information are available for free in an online appendix, if a reader has any questions the book did not answer. I strongly recommend this book and hope that it helps to improve awareness and discourse around this topic. Ideally, this might improve public debate on these risks and political action to prevent anyone from building a superintelligent AI, so that we and our children can have long and happy lives, and not have everyone die.