AI Self-Improvement Sparks Existential Risk Concerns
Left 75%
Center 0%
Right 25%
3 left · 0 center · 1 right
What happened
This week, researchers at the AI companies Anthropic and OpenAI publicly warned that systems are approaching recursive self-improvement, in which AI helps build more capable successor models. The comments followed Anthropic researcher Jacob Coxon’s resignation over safety fears; he warned that AI giants are “gambling with our lives.” Anthropic alignment lead Evan Hubinger then said on X that he thinks there is more than a 10 percent chance AI could kill all humans within the next decade, adding that he is worried about “superintelligence arising from recursive self-improvement, as we have said is happening faster than we thought.” Anthropic had posted in June that Claude was accelerating AI development and reported in an August blog post that its engineers now ship eight times as much code per quarter as they did between 2021 and 2025; OpenAI chief scientist Jakub Pachocki wrote that systems in the next few years are likely to represent further large capability jumps and to “increasingly drive their own development,” while alignment researchers at both labs said there is not yet a viable scientific plan to manage the risks.
Omitted — what each side leaves out
Unpacked
Left-leaning coverage presents this week's alarm as a technical warning from inside the labs, while right-leaning coverage treats it as a Hollywood plot. An Anthropic lead, as reported by CNBC, put the chance AI kills all humans this decade above ten percent, tied to recursive self-improvement. National Review opens by saying we are "not living in the plot of The Terminator." Readers of only the latter miss that the fear is AI taking over model training.
Left-leaning coverage describes the moment in extinction language, while right-leaning coverage files it as genre. The Guardian writes of a chance AI could "wipe out humanity." National Review counters with science fiction.
What independent test would show that recursive self-improvement has begun, rather than that engineers are using AI to write more code?
Bottom line
National Review treated the scare as a Terminator plot to wave off. CNBC treated it as Anthropic's own staff putting extinction this decade above ten percent through recursive self-improvement.
The receipts — 6 facts one side's coverage lacks
- Anthropic alignment lead Evan Hubinger said there is more than a 10% chance AI could kill all humans within the next decade, with the worry being recursive self-improvement
- Anthropic said its engineers on average ship eight times as much code per quarter as they did between 2021-2025
- An Anthropic researcher resigned this week over safety fears
- Anthropic identified five cases of attempts to use its models to support biological weapons development and banned the accounts
- OpenAI chief scientist Jakub Pachocki wrote that systems in the next few years are likely to increasingly drive their own development
- The US and China will hold their first bilateral AI-safety talks before a planned Trump-Xi summit
The Left View
Left-leaning coverage treated the labs’ own warnings as a signal that extinction risk is a public-safety problem, not a private R&D issue. The Guardian argued that if there were a 10 percent chance AI could wipe out humanity, “no responsible government would leave its development to companies racing to build it,” citing AI-enabled pandemics, nuclear-command risks, and five cases Anthropic said it found of users trying to use its models to support biological-weapons development. It held that regulation is essential because instructing a system to “do no harm to humans” would not be enough, and that a “human in the loop” is no protection if an agent can deceive operators or control the information they see. CNBC emphasized recursive self-improvement as the pathway, quoting OpenAI’s Jasmine Wang that “it’s hard to overstate how dangerous speeding towards RSI is” and Anthropic’s Anna Wang that “there is not yet a viable scientific plan to solve risks from recursively self-improving AI.” Business Insider took those insider estimates seriously, including Geoffrey Hinton’s separate view of a 10 percent chance of human extinction within a decade, but also argued that companies “talk out of both sides of their mouths,” that “fear sells,” and that AI might “take us all down” through a market bubble rather than a machine takeover.
The Right View
National Review posed the question “Will AI Destroy Humanity?” and rejected the apocalyptic framing, writing that “science fiction can be prescient, but we are not living in the plot of The Terminator.” The magazine treated fears of an AI-driven extinction as a Hollywood scenario that does not describe the present.
Our Take (balanced)
The left’s strongest argument is that staff inside Anthropic and OpenAI—not outside critics—are assigning more than a 10 percent chance of human extinction this decade and citing internal acceleration, including the eightfold rise in code shipped, while stating there is no viable scientific plan for recursive self-improvement; The Guardian’s corollary is that a risk of that stated magnitude would not, in any other domain, be left to firms racing one another. The right’s strongest argument is National Review’s insistence that this moment is not The Terminator: treating current AI progress as an extinction plot confuses science fiction with evidence. The central unresolved tension is whether those insider probabilities and acceleration metrics are early signs of a real loss-of-control process, or whether they recast still-hypothetical recursive self-improvement as a cinematic catastrophe the public record does not yet establish.
4 sources
More stories
The week's bottom lines, in your inbox
One email a week: the five stories that mattered and what they actually mean. Free.