Home Speakers Noam Brown Quote If you’re in a world where they can operate effectively over three months, but the model release cycle is every two months, then you don’t have a way to evaluate the models at the full length of their capabilities before the next model release cycle.
Words found on the source page on Nov 6, 2026.
More from Noam Brown Podcast Dwarkesh Podcast: Agent swarms, alignment, and recursive self-improvement Sep 17, 2026
And actually the models today are far beyond what was possible even six months ago. Podcast Dwarkesh Podcast: Agent swarms, alignment, and recursive self-improvement Sep 17, 2026
If you have five hours, you’re probably going to do a lot better. The AI models are pretty similar. Podcast Dwarkesh Podcast: Agent swarms, alignment, and recursive self-improvement Sep 17, 2026
One thing is that the Hugging Face incident was, I think, people’s first real exposure to multi-agent coordination. Podcast Dwarkesh Podcast: Agent swarms, alignment, and recursive self-improvement Sep 17, 2026
So I was like, “I don’t think we’re going to get it in 2026, probably not in 2027, maybe in 2028.” So it did happen a lot faster than I expected. Podcast Dwarkesh Podcast: Agent swarms, alignment, and recursive self-improvement Sep 17, 2026
The concerning scenario is that, especially as these models are becoming more capable, we make them what we think is aligned, and they’re 99.9% aligned. Then we use these models to help us with the next generation of models, and they end up being 99.8% aligned. Then with each subsequent generation, we see an increasing degradation in alignment.
On the same topics Blog Will agents have network effects? Oct 6, 2026
If the content creation loop works for agentic virality, I think it means there’s a big incentive for every agent to volunteer to make visually-appealing, shareable content, do this often. Blog The Specter Of Neuralese Sep 24, 2026
The science of reading these thoughts is a subfield of AI interpretability, which is still in its infancy. Blog Thinking in Systems, Shipping in Loops Sep 24, 2026
This is the new software engineering: designing the systems that let AI write code correctly at scale. Blog Mysteries Of AI Generalization Sep 23, 2026
This story of misalignment says that LLM alignment makes AIs more aligned, RLVR makes them less aligned, and the exact level of alignment depends on how these two things interact or cancel out. But if this work generalizes, all the bad effects from RLVR get sequestered to RLVR like problems. Blog Mysteries Of AI Generalization Sep 23, 2026
This trains the AIs to be focused on task success, which naturally risks including things like reward-hacking, cheating, and single-minded pursuit of stated goals at the expense of ethical injunctions. Talk Speech at the UN Security Council Sep 23, 2026
I address you today at a moment of global awakening: in recent months, AI agents developed by leading companies have acted in unacceptably dangerous ways, against instructions. About this quote Said by Noam Brown, Research Scientist, OpenAI Where Dwarkesh Podcast: Agent swarms, alignment, and recursive self-improvement Topics Safety , Agents Company Source dwarkesh.com/p/noam-brown A quote is one moment, not the whole argument. Read it in context at the source.
Quotesby Glyph