Should current AI development be significantly slowed or redirected to avoid recursive self-improvement and loss of control?
The debate is whether AI development should slow to avoid loss of control or continue advancing while managing risks. On the cautious side, Kokotajlo (Joe Rogan) and Hastings-Woodhouse (Doom Debates) argue for restraining or redirecting paths toward superintelligence and recursive self-improvement. On the faster side, DHH (Lex Fridman) and Huang (Hard Fork) emphasize embracing AI progress while handling concrete safety issues, in discussions since late August.
AI development should slow or avoid recursive self-improvement until safety is reliable.
Kokotajlo (Joe Rogan) argues AI work should proceed very cautiously and slowly to avoid worlds controlled by misaligned superintelligences. Hastings-Woodhouse (Doom Debates) says AI companies currently treat achieving recursive self-improvement as a goal when they should instead aim to avoid it. Wiblin (80,000 Hours Podcast) urges AI company staff to switch projects if their current work does not delay or safely shape recursive self-improvement. Kokotajlo later describes the aim as building aligned superintelligences whose goals and traits match what developers intend, rather than losing control.
3 claims, in their words
-
“We want to avoid the loss of control stuff. So we want to make it the case that we don't get the world taken over by misaligned superintelligences. Insofar as we're going to be building superintelligences at all, which we do in our, in our scenario and in our recommendation, we want to be doing it very cautiously and slowly,”
-
“it's still kind of the stated goal of all of the AI companies to just like do RSI. When in fact, they should be like, we really want to avoid doing RSI.”
-
“I have a request of all AI company staff. Reflect today on whether the projects you're working on is the best thing you can plausibly be doing to either delay recursive self-improvement or make it go well. And if it's not, push to be moved to an alternative project you think is more valuable.”
AI development should continue pursuing capabilities while addressing risks without major slowdown.
DHH (Lex Fridman) tells programmers to lean into industry changes brought by AI, using a regret minimization frame about how they will feel in a few years. Huang (Hard Fork) argues developers should first improve practical containment and isolation of AI systems before adding new regulations for hypothetical problems. Sacks (All-In) says the next year or so should downplay talk of sentience or extinction and instead highlight uses like curing cancer and improving lives. Huang and Sacks both frame AI progress as something to demonstrate concrete benefits and manage known risks around deployment, not as a project to pause broadly.
3 claims, in their words
-
“I also think that you're able to discover new sides of yourself if you let it in. Again, I keep coming back to this almost regret minimization framework, to borrow Jeff Bezos's terms here. Are you going to look at yourself two years from now and think, oh, I spent my time well being pissy about the present, about the”
-
“before we go fix the hypothetical problems, before we go create more regulations, can we work on the practical problems that we know exist, which is we need to do a better job with containment and isolation, which is we should not allow a product to interact with the external world until it's ready to be interacting with external worlds.”
-
“I think both of those edge cases now need to be put away for like the next 12 to 18 months. Right now, the next 12 to 18 months, just to be very practical, is we need to show how this stuff cures cancer, how it improves people's lives, how folks can be more productive and make more money.”
Listeners of “AI development should continue pursuing capabilities while addressing risks without major slowdown.” have not heard
Listeners of these shows have not heard Kokotajlo's warning that building superintelligences without great caution risks losing control to misaligned systems. They have not heard Hastings-Woodhouse's claim that AI companies currently treat achieving recursive self-improvement as their goal instead of explicitly aiming to avoid it. They have not heard Wiblin's request that AI staff reconsider their roles and move if needed to work that delays or safely shapes recursive self-improvement.
3 unanswered arguments, in their words
-
“it's still kind of the stated goal of all of the AI companies to just like do RSI. When in fact, they should be like, we really want to avoid doing RSI.”
-
“I have a request of all AI company staff. Reflect today on whether the projects you're working on is the best thing you can plausibly be doing to either delay recursive self-improvement or make it go well. And if it's not, push to be moved to an alternative project you think is more valuable.”
-
“We want to avoid the loss of control stuff. So we want to make it the case that we don't get the world taken over by misaligned superintelligences. Insofar as we're going to be building superintelligences at all, which we do in our, in our scenario and in our recommendation, we want to be doing it very cautiously and slowly,”
Listeners of “AI development should slow or avoid recursive self-improvement until safety is reliable.” have not heard
Listeners of these shows have not heard DHH's suggestion that individual programmers should adopt a regret minimization frame and actively embrace AI-driven changes to their work. They have not heard Huang's point that developers ought to prioritize better technical containment and isolation of AI systems before expanding regulations for hypothetical risks. They have not heard Sacks's claim that debates about AI sentience or extinction should be set aside temporarily to showcase medical and everyday benefits.
3 unanswered arguments, in their words
-
“before we go fix the hypothetical problems, before we go create more regulations, can we work on the practical problems that we know exist, which is we need to do a better job with containment and isolation, which is we should not allow a product to interact with the external world until it's ready to be interacting with external worlds.”
-
“I also think that you're able to discover new sides of yourself if you let it in. Again, I keep coming back to this almost regret minimization framework, to borrow Jeff Bezos's terms here. Are you going to look at yourself two years from now and think, oh, I spent my time well being pissy about the present, about the”
-
“I think both of those edge cases now need to be put away for like the next 12 to 18 months. Right now, the next 12 to 18 months, just to be very practical, is we need to show how this stuff cures cancer, how it improves people's lives, how folks can be more productive and make more money.”
Cautious arguments rest on avoiding loss of control from misaligned superintelligences and delaying recursive self-improvement. Capability arguments rest instead on programmers embracing AI, improving containment and isolation, and highlighting near-term benefits like medical advances.
- Nov 8, 2026 1 new claim (Hard Fork).
- Nov 6, 2026 2 new claims (80,000 Hours Podcast).
- Nov 4, 2026 Published: cleared the gate.
- Nov 4, 2026 6 new claims (All-In, 80,000 Hours Podcast, Lex Fridman Podcast, The Joe Rogan Experience, Doom Debates, Hard Fork).
Published because: 6 shows, 2 positions, 55 days of debate. Questions publish only when claims come from at least three shows, at least two positions are held by different people, and the debate has lasted two weeks.
Pairs are matched by content across different episodes and shows; no exchange between these speakers took place unless both quotes come from the same episode.
The prose on this page is written by a model from the quotes shown, and every sentence of it rests on quotes you can open; sentences that could not be traced to a quote were removed before publishing. Positions are placed from each speaker's own words. Evidence labels say what the speaker cited, never whether it is true.
9 episodes behind this page
- 2026-11-06Hard Fork — Why the Tech Right Lost Big This Week
- 2026-11-0580,000 Hours Podcast — An AI disaster hits. The president calls Xi. Then what? | Jeremie Harris (crosspost from NonZero)
- 2026-10-2180,000 Hours Podcast — Has your project been overtaken by events? 8 strategic updates | Rob Wiblin
- 2026-10-10All-In — Is Claude Conscious? Pope Rejects, Model Welfare Movement, OpenAI's Math Backlash, France Riots
- 2026-10-0880,000 Hours Podcast — In 2023 Ajeya Cotra already knew what was coming (classic episode)
- 2026-10-07Doom Debates — AI Has Become Profoundly WEIRD, But Is Anyone Noticing? With Sarah Hastings-Woodhouse
- 2026-09-25Hard Fork — The Ezra Klein Show: Jensen Huang Thinks A.I. Alarmism Has Gone Too Far
- 2026-09-09The Joe Rogan Experience — #2551 - Daniel Kokotajlo
- 2026-08-26Lex Fridman Podcast — #501 – DHH: Future of Programming, AI, Agentic Engineering, Vibe Coding & Linux
The week's bottom lines, in your inbox
One email a week: the stories that mattered and what they actually mean. Free.