Debate Over Pacing and Other Ways to Avoid AI Doom

In the Guardian, AI researcher David Krueger argues that pacing will not remove extinction risk. He writes that “we don’t know how these AI systems work,” how to keep them from misbehaving, how to “stay in control once they’re smarter than us,” or how to stop them from “playing nice or playing dumb to trick our safety tests,” and that another year or two of research cannot be counted on to solve those problems. He calls for “an immediate, indefinite, international moratorium on frontier AI development,” enforced by decommissioning the advanced chips and data centers needed to train frontier models and by dismantling the concentrated EUV and DUV lithography supply chain—steps he says would be easier to verify than coordination schemes that would require “increasing levels of surveillance.” Krueger characterizes Amodei’s path as still aiming at “recursive self-improvement,” “a project designed to render humans obsolete,” and treats invited outside evaluators as “more self-regulation” under a “safe-until-proven dangerous” standard “guaranteed to fall far short” of industries such as aviation. Economic costs, he argues, are “clearly worth it” given the stakes, analogizing to Covid-19 shutdowns, and he urges the public to “demand a plan that makes us safe, not just safer” rather than “listening to the AI industry about how it wants to be regulated.”