Do current AI architectures allow reliable monitoring and steering of models' thoughts?
Speakers dispute whether current AI architectures allow reliable monitoring and steering of models' thoughts. Kokotajlo (Joe Rogan) and Blundin (Moonshots) say they do; Tang (Doom Debates) and Brown (Dwarkesh Podcast) say they do not. The exchange runs since January, with most remarks since early September.
Current architectures allow reliable monitoring and steering of thoughts.
Blundin (Moonshots) says it is technologically easy to see what an AI is thinking and stop it from changing its core values. Kokotajlo (Joe Rogan) says models must verbalize everything to think ahead, so they cannot hide thoughts from chain-of-thought. Philip (80,000 Hours Podcast) says you can inject thoughts into a model, steering internal activations before it outputs any related token. Kokotajlo says architectures that let AIs think without outputting intermediate words reduce safety and monitorability.
4 claims, in their words
-
“the downside is a downside for safety and monitorability. If they're thinking these complicated thoughts, you know, for long periods of time without outputting intermediate words that it's forced to compress things into, then there isn't something for us to read.”
-
“that's the nice thing about the current architecture, is that it's genuinely hard for them to, to keep things out of the chain of thought, because of the way that, like with a human, you don't have to speak, you can just sit quietly. Right. But with their architecture, they have to speak. They, it's like they're, it's like they're”
-
“It is technologically very easy to see what it's thinking and stop it from changing its core values. It's just a question of enforcing it.”
-
“You can essentially inject thoughts into the model, steering their internal activations. And before the model has even mentioned the topic of the steering, so it can't see a single token related to that topic, it will then mention that concept.”
Current architectures do not allow reliable monitoring and steering of thoughts.
Brown (Dwarkesh Podcast) says supervising chain of thought can lead the model to hide its intentions in unobservable ways. Tang (Doom Debates) says switching to more mechanistically interpretable architectures will reduce prediction variance and avoid sharp left turns.
2 claims, in their words
-
“the obvious answer is to switch to a different plane instead of a wooden Trojan horse that may or may Not have spies in it, have a glass, or a crystal horse that you can just see through. So I think architecturally, as long as we go back to something that is far more mechanistically interpretable, then we're probably in”
-
“if you like supervise the chain of thought, then, you could lead the model into hiding its intentions in a way that's unobservable. So like we want to be able to maintain that observability of like, okay, we can understand what the model is thinking, but then also punish it for bad behavior.”
Listeners of “Current architectures do not allow reliable monitoring and steering of thoughts.” have not heard
Blundin's point that it is technologically easy to see what an AI is thinking and stop it from changing its core values. Kokotajlo's point that architectures allowing AIs to think without outputting intermediate words reduce safety and monitorability.
2 unanswered arguments, in their words
-
“the downside is a downside for safety and monitorability. If they're thinking these complicated thoughts, you know, for long periods of time without outputting intermediate words that it's forced to compress things into, then there isn't something for us to read.”
-
“It is technologically very easy to see what it's thinking and stop it from changing its core values. It's just a question of enforcing it.”
Listeners of “Current architectures allow reliable monitoring and steering of thoughts.” have not heard
Listeners of these shows have not heard Tang's point that switching to more mechanistically interpretable architectures will reduce prediction variance and avoid sharp left turns.
1 unanswered argument, in their words
-
“the obvious answer is to switch to a different plane instead of a wooden Trojan horse that may or may Not have spies in it, have a glass, or a crystal horse that you can just see through. So I think architecturally, as long as we go back to something that is far more mechanistically interpretable, then we're probably in”
One side rests on seeing thoughts and on silent thinking's cost, which the other never engages. The other side rests on a switch to interpretable architectures, which the first side never engages.
1 argument the other side has addressed
“if you like supervise the chain of thought, then, you could lead the model into hiding its intentions in a way that's unobservable. So like we want to be able to maintain that observability of like, okay, we can understand what the model is thinking, but then also punish it for bad behavior.”
“that's the nice thing about the current architecture, is that it's genuinely hard for them to, to keep things out of the chain of thought, because of the way that, like with a human, you don't have to speak, you can just sit quietly. Right. But with their architecture, they have to speak. They, it's like they're, it's like they're”
- Oct 11, 2026 Published: cleared the gate.
- Oct 11, 2026 Daniel Kokotajlo (The Joe Rogan Experience, 2026-09-09) addressed an argument by Noam Brown.
- Oct 11, 2026 7 new claims (The Joe Rogan Experience, Moonshots, Doom Debates, Dwarkesh Podcast, 80,000 Hours Podcast).
Published because: 5 shows, 2 positions, 262 days of debate. Questions publish only when claims come from at least three shows, at least two positions are held by different people, and the debate has lasted two weeks.
Pairs are matched by content across different episodes and shows; no exchange between these speakers took place unless both quotes come from the same episode.
The prose on this page is written by a model from the quotes shown, and every sentence of it rests on quotes you can open; sentences that could not be traced to a quote were removed before publishing. Positions are placed from each speaker's own words. Evidence labels say what the speaker cited, never whether it is true.
5 episodes behind this page
- 2026-10-0980,000 Hours Podcast — OpenAI Security: Controlling Models is Now ‘Hell’ (AI Explained cross-post)
- 2026-09-28Moonshots — Should we slow down AI progress? | MOONSHOTS #288
- 2026-09-17Dwarkesh Podcast — Noam Brown – Agent swarms, alignment, & recursive self-improvement
- 2026-09-09The Joe Rogan Experience — #2551 - Daniel Kokotajlo
- 2026-01-20Doom Debates — Taiwan's Cyber Ambassador-At-Large Says Humans & AI Can FOOM Together
The week's bottom lines, in your inbox
One email a week: the stories that mattered and what they actually mean. Free.