AI Leaders Debate the Probability of Human Extinction
Silicon Valley's p(doom) estimates reveal deep divisions on AI's existential risks.
1P(doom) emerges as key metric
AI researchers at leading labs like Anthropic and OpenAI have begun quantifying existential risks through p(doom) - the estimated probability that AI could cause human extinction within a decade. Figures range dramatically from 5% to 80%, with Anthropic's Evan Hubinger recently stating a >10% chance. This metric has sparked intense debate despite its inherently speculative nature, reflecting growing concerns about advanced AI systems.
2Wide variance in expert estimates
Prominent figures disclose wildly different p(doom) assessments: Anthropic CEO Dario Amodei estimates 10-25%, while former OpenAI researcher Daniel Kokotajlo believes 70%. The variation underscores the subjective nature of these predictions, as no mathematical model exists to calculate such risks accurately. These numbers represent personal intuitions rather than scientific calculations about future AI capabilities and their potential consequences.
3Critics call it fear-mongering
Some experts argue p(doom) discussions serve as marketing tactics ahead of major IPOs from OpenAI and Anthropic. Computer scientist Cal Newport condemns the 'doom trolling' as emotionally manipulative, suggesting it creates unnecessary public anxiety while benefiting companies positioning themselves as humanity's potential saviors against existential AI threats.
4Researchers defend genuine concerns
Anthropic's Drake Thomas counters that these fears reflect real apprehension among AI safety teams. The debate reveals a tension between acknowledging legitimate risks and avoiding sensationalism. Many researchers genuinely worry about controlling increasingly powerful systems, even as their warnings contribute to the industry's mystique.
5Strategic narrative of inevitability
AI companies frame superintelligence as an unavoidable evolutionary step, using p(doom) to justify continued development. They argue that not building advanced AI would cede advantage to geopolitical rivals like China. This narrative positions them as necessary stewards of dangerous technology, creating a paradox where the perceived threat becomes the rationale for acceleration.
6Public trust erosion
The combination of apocalyptic warnings and claims that only AI companies can prevent disaster creates skepticism. As OpenAI's Sam Altman admits difficulty quantifying p(doom), the public struggles to reconcile contradictory messages about AI's benefits versus existential risks. This tension may undermine confidence in the industry's ability to self-regulate.
7No scientific basis for estimates
All p(doom) figures represent subjective guesses rather than calculated probabilities. The complex, unprecedented nature of advanced AI makes reliable forecasting impossible. Presenting personal intuitions as percentages lends false precision to fundamentally uncertain predictions about future technological developments and their societal impacts.
8Broader implications for AI governance
The p(doom) debate highlights urgent questions about responsible AI development. Without credible risk assessment methods, subjective probabilities may drive policy and investment decisions. This situation demands more transparent frameworks for evaluating and communicating AI risks that balance precaution with innovation.
Original article: Gizmodo — AI Leaders Debate the Probability of Human Extinction