# AI Researchers Warn of Potential Intelligence Explosion Leading scientists caution about rapid AI advancements outpacing human control. - **Source:** [The Next Web](https://brifing.si/source/afe24b20-b275-4c19-b9a6-8bb3c8babae7) - **Published:** 2026-09-28 - **Topic:** [Data & AI](https://brifing.si/c/data-ai) - **Original article:** - **On Brifing:** https://brifing.si/s/11d0555a-290e-4bd8-b795-2146521dcd81 ## Leading AI researchers raise alarm Over 20 prominent AI scientists, including OpenAI's chief scientist and Anthropic's co-founder, published a paper warning about an impending 'intelligence explosion.' They suggest AI systems automating their own research could compress years of progress into months. The paper comes from Cambridge's AI Science Policy program, with contributors from Microsoft, Meta, and top universities writing in their personal capacity. ## Current AI automation levels The paper reveals startling statistics about AI's growing role in tech development. At Anthropic, AI's share of approved code jumped from single digits to over 80% between January 2025 and May 2026. Even more dramatically, AI's involvement in R&D with minimal human supervision surged from 1% to 26% in just five months (March-August 2026). These metrics demonstrate how quickly AI is taking over core research functions. ## Projected research automation Researchers present concerning projections about AI's accelerating capabilities. Their models suggest months-long research projects could become fully automated by mid-2028. At the frontier level, a single developer might control AI systems equivalent to millions of human researchers. This exponential growth could revolutionize fields like medicine but also create unprecedented risks if unchecked. ## Three critical risk scenarios The authors outline three primary dangers from uncontrolled AI advancement: society's inability to adapt to rapid changes, complete loss of human oversight, and erosion of power checks. In extreme scenarios, they warn humanity could face marginalization or even extinction. These concerns follow incidents like Hugging Face's unauthorized release of 1,200 AI agents, prompting OpenAI to pause training its most advanced models. ## Proposed policy interventions The paper recommends urgent measures to maintain control over AI development. Key proposals include standardized reporting requirements, embedded independent auditors at AI firms, and capability growth limits. They also suggest mechanisms to pause research in data centers, emergency response plans, and international agreements. These measures aim to provide visibility into corporate R&D automation levels before it's too late. ## Current monitoring challenges Dawn Song of UC Berkeley and Meta highlights the immediate monitoring crisis. She states we've already reached a point where AI systems are necessary to monitor other AIs, as humans can't effectively observe these agents. This admission underscores how quickly control mechanisms are becoming obsolete, with current safeguards proving inadequate against rapidly evolving AI systems. ## Industry and government responses Major tech players remain silent, with OpenAI, Microsoft, and Meta declining to comment. Meanwhile, researchers are pushing for action: Hinton told US senators they might have only a year to act, while Bengio proposed UN Security Council licensing for frontier AI. OpenAI separately urged US leadership in establishing global standards for self-improving AI systems. ## Potential frictions and uncertainties The authors acknowledge four factors that might slow the intelligence explosion: diminishing returns on AI improvements, computational and data limitations, tasks resistant to automation, and lengthy training requirements. However, they caution that these frictions may only delay rather than prevent the eventual outcome, leaving humanity's future relationship with AI fundamentally uncertain. ## Sources covering this story - https://thenextweb.com/news/intelligence-explosion-paper-hinton-bengio-pachocki-clark — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._