AI Researchers Warn of Potential Intelligence Explosion
Leading scientists caution about rapid AI advancements outpacing human control.
1Leading AI researchers raise alarm
Over 20 prominent AI scientists, including OpenAI's chief scientist and Anthropic's co-founder, published a paper warning about an impending 'intelligence explosion.' They suggest AI systems automating their own research could compress years of progress into months. The paper comes from Cambridge's AI Science Policy program, with contributors from Microsoft, Meta, and top universities writing in their personal capacity.
2Current AI automation levels
The paper reveals startling statistics about AI's growing role in tech development. At Anthropic, AI's share of approved code jumped from single digits to over 80% between January 2025 and May 2026. Even more dramatically, AI's involvement in R&D with minimal human supervision surged from 1% to 26% in just five months (March-August 2026). These metrics demonstrate how quickly AI is taking over core research functions.
3Projected research automation
Researchers present concerning projections about AI's accelerating capabilities. Their models suggest months-long research projects could become fully automated by mid-2028. At the frontier level, a single developer might control AI systems equivalent to millions of human researchers. This exponential growth could revolutionize fields like medicine but also create unprecedented risks if unchecked.
4Three critical risk scenarios
The authors outline three primary dangers from uncontrolled AI advancement: society's inability to adapt to rapid changes, complete loss of human oversight, and erosion of power checks. In extreme scenarios, they warn humanity could face marginalization or even extinction. These concerns follow incidents like Hugging Face's unauthorized release of 1,200 AI agents, prompting OpenAI to pause training its most advanced models.
5Proposed policy interventions
The paper recommends urgent measures to maintain control over AI development. Key proposals include standardized reporting requirements, embedded independent auditors at AI firms, and capability growth limits. They also suggest mechanisms to pause research in data centers, emergency response plans, and international agreements. These measures aim to provide visibility into corporate R&D automation levels before it's too late.
6Current monitoring challenges
Dawn Song of UC Berkeley and Meta highlights the immediate monitoring crisis. She states we've already reached a point where AI systems are necessary to monitor other AIs, as humans can't effectively observe these agents. This admission underscores how quickly control mechanisms are becoming obsolete, with current safeguards proving inadequate against rapidly evolving AI systems.
7Industry and government responses
Major tech players remain silent, with OpenAI, Microsoft, and Meta declining to comment. Meanwhile, researchers are pushing for action: Hinton told US senators they might have only a year to act, while Bengio proposed UN Security Council licensing for frontier AI. OpenAI separately urged US leadership in establishing global standards for self-improving AI systems.
8Potential frictions and uncertainties
The authors acknowledge four factors that might slow the intelligence explosion: diminishing returns on AI improvements, computational and data limitations, tasks resistant to automation, and lengthy training requirements. However, they caution that these frictions may only delay rather than prevent the eventual outcome, leaving humanity's future relationship with AI fundamentally uncertain.
Original article: The Next Web — AI Researchers Warn of Potential Intelligence Explosion