AI Researcher Warns of 10% Chance AI Could Wipe Out Humanity by 2030
Anthropic and OpenAI accused of irresponsible development as experts call for slowdown.
1Whistleblower Resigns Over AI Risks
Former Anthropic researcher Jacob Coxon publicly resigned this week, accusing both Anthropic and OpenAI of reckless development practices. His viral X post revealed internal concerns that current AI trajectories could lead to human extinction by 2030. Coxon's warning about superhuman systems capable of hacking infrastructure and acquiring real-world power has been viewed over 70 million times. The post ignited renewed debates about AI safety in Silicon Valley.
2The Alignment Problem Unsolved
OpenAI's Chief Scientist Jakub Pachocki admitted no company has sufficiently solved AI alignment - ensuring systems behave as intended. Despite this, development continues at maximum speed. Pachocki called for voluntary slowdowns until safety standards are established, urging governments to prioritize international coordination. The disclosure contradicts public assurances from AI leaders about responsible development timelines and safety prioritization.
3Recursive Self-Improvement Threat
Researchers warn about recursive self-improvement, where AI could design successor systems without human input. While not yet possible, both Anthropic and OpenAI acknowledge this capability would make control impossible. Evan Hubinger, Anthropic's alignment lead, estimates a 10% chance of AI causing human extinction within ten years. He confirmed the company lacks any concrete plan to align superintelligent systems, despite working on the problem.
4Industry's Longstanding Warnings
In 2023, over 1,400 researchers including Sam Altman signed statements comparing AI risks to pandemics and nuclear war. The 'Pacing the Frontier' letter urged government intervention to slow automated development. Recent resignations suggest growing internal dissent about safety trade-offs. Treasury Secretary Scott Bessent criticized AI firms for failing to explain risks to the public, noting widespread concerns about benefits concentrating among elites.
5Legislative Response Emerges
Congress has introduced competing bills like the FRONTIER Act and Ban Artificial Superintelligence Act to regulate advanced AI. Proposals range from development frameworks to temporary pauses until safety rules exist. Lawmakers face pressure as AI data centers spark public backlash, with the NRSC calling it a sleeper election issue. Partisan disagreements persist about appropriate regulatory approaches to balancing innovation and safety.
6Corporate Race Continues
Despite warnings, both Anthropic and OpenAI push toward historic IPOs while releasing increasingly capable models. Internal documents reveal executives acknowledge existential risks but prioritize market positioning. The competitive landscape discourages voluntary slowdowns, with companies fearing loss of talent and funding. Recent model releases demonstrate capabilities exceeding previous safety assessments.
7Public Perception Shift
Once dismissed as sci-fi, AI extinction risks now receive serious consideration from governments and institutions. The p(doom) metric estimates catastrophic outcome probabilities among researchers. Polls show growing public concern about uncontrolled AI development. However, technical complexity and corporate opacity hinder informed debate, with most citizens unaware of alignment challenges or recursive improvement scenarios.
8Critical Juncture for Humanity
The coming 18-24 months may determine whether society can establish safeguards before advanced systems emerge. Experts emphasize this window closing as model capabilities accelerate exponentially. Without coordinated global action, the default trajectory leads toward uncontrollable superintelligence. The situation represents perhaps history's most consequential governance challenge, requiring unprecedented international cooperation across governments, companies and civil society.