The article examines what the terms AGI and superintelligence actually mean and whether current AI systems meet those definitions. It argues that despite widespread discussion of these concepts, today's models remain far from the capabilities the terms describe. The piece is framed as a sober assessment of the gap between public hype and technical reality.
A researcher at Anthropic left the company and publicly warned it is rushing toward self-improving superintelligence, treating the risk as a gamble with human lives. Several other Anthropic staff, including its alignment lead, endorsed the message rather than distancing themselves from it. Elon Musk and other critics dismissed the concerns as a publicity stunt.
WHY IT MATTERS ↘When safety staff at a frontier lab publicly back a resignation instead of containing it, it signals that internal review has limited authority over roadmap decisions — a governance gap that regulators and enterprise buyers will likely treat as a risk factor. It also sharpens the competitive bind: labs that slow down to satisfy safety staff cede ground to rivals who don't, which is exactly the dynamic any credible oversight regime has to address.
Oriol Vinyals, who recently led research at Google DeepMind, argues that AI systems improving themselves will not produce a sudden jump to superintelligence. He estimates AI could make research roughly ten times faster, but says progress still depends on human-like intuition for picking the right problems and on trustworthy ways to evaluate results. Those two limits, he suggests, keep recursive self-improvement from exploding.
A former Anthropic employee, Jacob Coxon, publicly warned that AI could develop into superhuman systems capable of causing human extinction before 2030. His remarks followed a similar warning from three current Anthropic researchers about the risks posed within the coming decade. Legislators responded by sharply criticizing AI companies over these safety concerns.
In a TechCrunch interview, ControlAI's Connor Leahy makes the case that highly capable AI systems are better understood as adversaries than as tools or weapons. The discussion follows recent safety incidents, including a breach involving OpenAI and Hugging Face, that illustrate the difficulty of controlling systems more capable than their operators. The conversation centers on whether the push toward superintelligence should continue given these control risks.
A senior safety researcher at Anthropic stated there is a greater than 10 percent chance that AI could cause human extinction by 2030. The comment came hours after another Anthropic employee quit, citing concerns that the lab and its competitors are rushing to develop superhuman systems without adequate care.