Is scaling AI model sizes and using current training-based alignment a broadly reliable way to improve AI safety?
The debate asks whether making AI models larger and using today’s training-based alignment methods reliably makes systems safer overall or instead increases risks. Cotra (80,000 Hours Podcast) argues that larger models more easily follow instructions and side constraints, so scaling plus current training is, for her, a broadly reliable path to greater safety. Vassar (Doom Debates) and Blundin (Moonshots) instead claim that pushing model sizes and relying on training-based alignment makes us less safe, especially as self-improving models emerge.
Yes, scaling and current training-based alignment reliably improve overall AI safety.
Cotra (80,000 Hours Podcast) says evidence from newer models shows that simply making them larger improves their ability to follow instructions and respect side constraints more than targeted alignment tricks. Cotra reports running the same prompts researchers used to motivate new alignment techniques on later, larger models and finding that these bigger systems often handle the tasks without special methods. Cotra argues that many proposed alignment methods mainly close capabilities gaps, such as poor instruction following, that scaling alone tends to fix. Cotra claims that getting models to follow user instructions is currently easy and becomes easier as model size increases.
3 claims, in their words
-
“I still see a lot of machine learning researchers that are focused on addressing what feel more like capabilities gaps than alignment gaps that do help the models act better, but are more like, oh, our systems aren't very good at following instructions. Let's kind of put them in a setup that makes them better at understanding instructions. And the”
-
“we have some pretty strong evidence that just making the models bigger helps with this. And in fact, I often get proposals actually from people who are like, models aren't able to do this kind of thing. So they're not able to follow side constraints very well. And there'll be some examples. But then when I run those exact questions”
-
“I think we're in a world where it's pretty easy to get them to do that, and as they get bigger, it's even easier.”
No, scaling and current training-based alignment do not reliably improve overall AI safety.
Vassar (Doom Debates) claims that pushing model sizes and using most current alignment research and training makes AI systems less safe, except for rare interpretability work by a few researchers. Wright (Doom Debates) argues that alignment methods will not keep pace as companies release models faster and those models start designing or training successors. Blundin (Moonshots) states that newly released open-source Chinese models can already improve themselves, so alignment based on training is no longer a viable primary safety strategy. Blundin says that in a world of self-improving models, safety must rely on inspection and containment rather than on more training-based alignment.
3 claims, in their words
-
“we don't need to push model sizes, we don't need to do the stupid alignment research and stupid alignment training that we do. These things are basically known to make us less safe, not more safe, except for the incredibly rare interpretability work that comes from, like, ten people.”
-
“as the models are coming out faster and faster and faster, and more and more are just making themselves, that's when We're going to catch up with the alignment challenge? I don't think so.”
-
“the new Chinese models are absolutely capable of self-improvement. There's no doubt in my mind. So that cat is out of the bag. It's, the boat has sailed. Forget it. As a practical matter, it's about inspection and containment that way going forward, not about training for alignment.”
Listeners of “No, scaling and current training-based alignment do not reliably improve overall AI safety.” have not heard
Listeners of these shows have not heard Cotra's report that larger later models often succeed on the same prompts earlier researchers used to justify new alignment techniques. Cotra's claim that scaling alone can make models better at following side constraints than specialized alignment methods has not appeared on these programs. Cotra's view that getting models to follow instructions is already easy and becomes easier as they grow has likewise not been discussed.
3 unanswered arguments, in their words
-
“I think we're in a world where it's pretty easy to get them to do that, and as they get bigger, it's even easier.”
-
“we have some pretty strong evidence that just making the models bigger helps with this. And in fact, I often get proposals actually from people who are like, models aren't able to do this kind of thing. So they're not able to follow side constraints very well. And there'll be some examples. But then when I run those exact questions”
-
“I still see a lot of machine learning researchers that are focused on addressing what feel more like capabilities gaps than alignment gaps that do help the models act better, but are more like, oh, our systems aren't very good at following instructions. Let's kind of put them in a setup that makes them better at understanding instructions. And the”
Listeners of “Yes, scaling and current training-based alignment reliably improve overall AI safety.” have not heard
Listeners of these shows have not heard Vassar's claim that increasing model sizes and most current alignment training makes systems less safe, except for a little interpretability work. Wright's point that alignment techniques will fall further behind as models are released faster and start contributing to their own development is also absent from these shows. Blundin's argument that self-improving open-source Chinese models make training-based alignment unworkable and shift the focus to inspection and containment is likewise missing.
3 unanswered arguments, in their words
-
“as the models are coming out faster and faster and faster, and more and more are just making themselves, that's when We're going to catch up with the alignment challenge? I don't think so.”
-
“we don't need to push model sizes, we don't need to do the stupid alignment research and stupid alignment training that we do. These things are basically known to make us less safe, not more safe, except for the incredibly rare interpretability work that comes from, like, ten people.”
-
“the new Chinese models are absolutely capable of self-improvement. There's no doubt in my mind. So that cat is out of the bag. It's, the boat has sailed. Forget it. As a practical matter, it's about inspection and containment that way going forward, not about training for alignment.”
Scaling proponents lean on Cotra's observations about larger models' improved instruction-following and constraint-handling, while critics' shows never directly address these scaling-based safety gains. Skeptics center their case on Vassar's and Blundin's claims that bigger, self-improving models plus current training make systems less safe, which scaling advocates' shows do not confront.
- Nov 7, 2026 Published: cleared the gate.
- Nov 7, 2026 Dave Blundin moved from “AI models are improving, releasing, and getting cheaper at an accelerating rate.” to “No, scaling and current training-based alignment do not reliably improve overall AI safety.”.
- Nov 7, 2026 5 new claims (80,000 Hours Podcast, Doom Debates, Moonshots).
- Nov 6, 2026 4 new claims (All-In, The Ezra Klein Show, Moonshots).
- Nov 2, 2026 10 new claims (All-In, 80,000 Hours Podcast, Dwarkesh Podcast, Doom Debates, Moonshots).
Published because: 3 shows, 2 positions, 64 days of debate. Questions publish only when claims come from at least three shows, at least two positions are held by different people, and the debate has lasted two weeks.
Pairs are matched by content across different episodes and shows; no exchange between these speakers took place unless both quotes come from the same episode.
The prose on this page is written by a model from the quotes shown, and every sentence of it rests on quotes you can open; sentences that could not be traced to a quote were removed before publishing. Positions are placed from each speaker's own words. Evidence labels say what the speaker cited, never whether it is true.
5 episodes behind this page
- 2026-10-21Moonshots — Chamath: AGI Is Here, Anthropic’s $100B Giveaway, Elon’s SpaceX Shares for 72M Kids | EP #304
- 2026-10-0880,000 Hours Podcast — In 2023 Ajeya Cotra already knew what was coming (classic episode)
- 2026-10-07Doom Debates — AI Has Become Profoundly WEIRD, But Is Anyone Noticing? With Sarah Hastings-Woodhouse
- 2026-10-02Doom Debates — Trump and the AI Safety Vibe Shift, With Robert Wright | Nonzero × Doom Debates
- 2026-08-18Doom Debates — Former Singularity Institute President: PauseAI Is Making AI Doom WORSE
The week's bottom lines, in your inbox
One email a week: the stories that mattered and what they actually mean. Free.