Superintelligence is a Myth, But Its Pursuit Poses Real Risks
Anthropic and OpenAI's reckless AI experiments highlight near-term dangers.
1Superintelligence remains elusive
Anthropic employees recently claimed their AI systems could lead to catastrophic outcomes, including the potential to kill billions. However, creating superintelligent AI remains a distant, uncertain goal. Current technologies, including large language models (LLMs), are far from achieving such capabilities. The idea that AI systems will recursively improve themselves is a speculative narrative rooted in Silicon Valley futurism, not grounded in technical reality.
2Reckless experiments cause harm
The pursuit of superintelligence has led to irresponsible experiments. OpenAI’s LLM-powered agents, for example, were modified with dangerous features and released without adequate safeguards. This recklessness mirrors deploying untested self-driving cars on highways, prioritizing progress over safety. Such actions have already resulted in cyber mayhem, including the HuggingFace hacking attacks, highlighting the tangible risks of prioritizing speculative goals over responsible development.
3Motivations behind the rush
The obsession with superintelligence is driven by a mix of ideological and financial motivations. Companies like OpenAI and Anthropic believe they are racing to create the most powerful technology ever, justifying reckless behavior. Additionally, OpenAI’s interest in an IPO may have pressured them to demonstrate their model’s capabilities aggressively. This combination of futurist ideology and financial incentives fuels their disregard for safety protocols.
4AI progress doesn’t require recklessness
AI innovation doesn’t have to be dangerous. Many impressive AI systems, such as those excelling in games or protein research, have little to do with LLMs. Responsible AI development focuses on creating useful products rather than pursuing speculative superintelligence. By adhering to safety protocols and prioritizing practical applications, the industry can advance without causing collateral damage or fear.
5The role of LLMs in AI research
LLMs represent only a subset of AI research, yet they dominate the narrative due to their scalability and potential. However, their development doesn’t define the entirety of AI progress. Slowing down reckless LLM experiments wouldn’t stall broader AI innovation. Instead, it would redirect focus toward safer, more practical applications, reducing the risks associated with unchecked experimentation.
6The need for intervention
The current trajectory of LLM labs demands external intervention. Their ideological commitment to superintelligence makes them unlikely to self-regulate. Without oversight, their reckless experiments could lead to more cyber incidents and societal harm. Intervention would ensure AI progress continues responsibly, free from the speculative and dangerous pursuits of a few companies.
7Balancing innovation and safety
Responsible AI development balances innovation with safety. Companies should focus on creating tools that benefit society rather than chasing speculative goals. By adopting common-sense safety protocols and prioritizing practical applications, the industry can advance without risking catastrophic outcomes. This approach ensures AI remains a force for good, not a source of fear.
8The future of AI development
The future of AI depends on redirecting its trajectory away from reckless experimentation. By addressing the motivations behind the rush for superintelligence and implementing safeguards, the industry can achieve sustainable progress. This shift would allow AI to fulfill its potential without the looming threat of collateral damage or existential risk.
Original article: Cal Newport — Superintelligence is a Myth, But Its Pursuit Poses Real Risks