AI researchers warn superintelligence poses existential threat
Industry insiders compare uncontrolled AI development to 'playing with god-like power'
1Industry insiders sound alarm
Current and former researchers from OpenAI, Google DeepMind, and Anthropic have released video interviews warning about AI superintelligence dangers. Geoffrey Irving, formerly of OpenAI and Google, estimates human extinction risk at 50%. The interviews were collected by nonprofit Palisade Research on frominside.ai, featuring a dozen AI experts. These warnings echo recent concerns from the 'AI safety' community about uncontrolled development.
2Extinction probability estimates
Researchers provided stark probability assessments about AI risks. Google DeepMind's Neel Nanda estimates at least 10% chance of human extinction from superintelligent AI, calling this 'ridiculously high'. The interviews reveal no consensus on solutions, highlighting the field's uncertainty. Researchers acknowledge the difficulty in defining 'AI safety' and appropriate responses, with no cohesive solution emerging from these warnings.
3God-like power concerns
Former OpenAI researcher Daniel Kokotajlo described superintelligent AI as potentially 'god-like powerful' with no known control methods. He emphasized this represents exactly the danger it sounds like and must be prevented. The warnings focus on AI systems that could self-improve beyond human comprehension or control. Researchers stress current inability to contain or direct such theoretical superintelligence.
4Motivations questioned
The videos address why researchers continue working on potentially dangerous technology. Google's Mary Phuong acknowledges skepticism about lab-funded researchers' objectivity. Neel Nanda claims he continues only because he believes his work reduces existential risks. Researchers confront the paradox that individual resignations wouldn't stop development, as companies would continue without them.
5Segmented discussion approach
The interview project organizes responses by theme, including whether concerns represent genuine risk or industry hype. This structure allows comparison of different researchers' perspectives on similar questions. The full interviews are available alongside these thematic compilations. The format highlights both consensus points and diverging opinions among AI safety experts.
6Industry context of warnings
These warnings emerge amid growing AI safety discussions in tech. The interviews follow recent reports about risks from self-improving AI systems. They coincide with companies like OpenAI pausing training of advanced models. The researchers represent both current industry employees and those who've left major labs, providing varied institutional perspectives.
7Control problem emphasized
Researchers consistently highlight the 'control problem' - humanity's inability to constrain superintelligent systems. They note current AI safety measures address narrow issues but not existential risks. The interviews suggest alignment research lags behind capability development. Experts warn we lack technical solutions to ensure superintelligence would remain beneficial or controllable.
8Call for cautious development
The researchers collectively urge more cautious AI development despite differing risk assessments. They advocate prioritizing safety research alongside capability advances. The warnings aim to spur serious consideration of long-term risks before they materialize. While not proposing specific solutions, they stress the unprecedented nature of superintelligence challenges.
Original article: The Verge — AI researchers warn superintelligence poses existential threat