Existential Risk and the Future of Humanity: Lessons from AI, Pandemics, and Nuclear Threats (Toby Ord, Author of "The Precipice")

Speaker A: The human story might just be beginning, but there are various threats to our continued existence. Some of these have been around forever, such as asteroid impacts, but some of them are threats of our own making, such as nuclear war. Speaker B: You put the odds at 1 in 6 that in this century we do incur the existential risk and humanity fails to live beyond this century. Have the last few years made you more or less worried about this? Speaker A: So another possibility is what gets called gradual disempowerment.

Suppose the AI systems never violate our rights. Maintain sufficient control over them, they don't break the law, but they're more successful than us over time at doing all the kinds of jobs that we do. Even if we get richer through trade with these systems, a higher and higher share of the wealth eventually accumulates in AI hands. Speaker C: Hey, I'm Mario, and this is The Generalist Podcast. As the saying goes, the future is already here. Speaker B: It's just not evenly distributed. Speaker C: Each episode, we sit down with the visionaries, builders, and thinkers who are already living in that future to help you see it earlier, understand it better, and capitalize on it.

Today, I'm speaking with Toby Ord, a senior researcher at Oxford University, one of the world's leading experts on existential risk, and author of the excellent book The Precipice. It's a thoughtful analysis of the greatest dangers to humanity's survival. From rogue asteroids to pandemics to unaligned artificial intelligence. These may sound like fanciful sci-fi problems, but there's actually good reason to believe they're extremely pressing, with Toby estimating humanity has a 1 in 6 chance of extinction this century. In my conversation with Toby, we discuss how AI risk has evolved since LLMs emerged, why the scientists working on the Manhattan Project —didn't stop developing nuclear bombs after Hitler died—and the Cold War lessons the