If you are relying on anything remotely like ‘automated alignment evals’ against potential superintelligences, I predict that you are rather cooked.
Words found on the source page on Nov 6, 2026.
More from Zvi Mowshowitz
BlogWe Must Pace The FrontierSep 14, 2026
If I was in charge of an alignment or interpretability team I would never run out of experiments to run and things to try, including having the time to properly talk to the models in the first place.BlogWe Must Pace The FrontierSep 14, 2026
Over the summer there was a step change, as Mythos and Astra started kicking off the early stages of recursive self-improvement (RSI).BlogWe Must Pace The FrontierSep 14, 2026
Remember that ‘they will probably say no’ is not a good argument for not trying, even if you are right that the odds are not so great.BlogWe Must Pace The FrontierSep 14, 2026
Ultimately, global coordination is The Way. We can and should do things without it, to enable coordination, and to show good faith, and because as things are we would be better off acting unilaterally, including because our Chinese competition is fast following.
On the same topics
BlogThe Future Is for Everyone: Muse for Small BusinessSep 29, 2026
Muse is one of the most important products we’ve built for the era of personal superintelligence.UnverifiedBlogThe Specter Of NeuraleseSep 24, 2026
The science of reading these thoughts is a subfield of AI interpretability, which is still in its infancy.BlogMysteries Of AI GeneralizationSep 23, 2026
This story of misalignment says that LLM alignment makes AIs more aligned, RLVR makes them less aligned, and the exact level of alignment depends on how these two things interact or cancel out. But if this work generalizes, all the bad effects from RLVR get sequestered to RLVR like problems.BlogMysteries Of AI GeneralizationSep 23, 2026
This trains the AIs to be focused on task success, which naturally risks including things like reward-hacking, cheating, and single-minded pursuit of stated goals at the expense of ethical injunctions.TalkSpeech at the UN Security CouncilSep 23, 2026
I address you today at a moment of global awakening: in recent months, AI agents developed by leading companies have acted in unacceptably dangerous ways, against instructions.TalkSpeech at the UN Security CouncilSep 23, 2026
The companies building the most powerful AI systems admit their products pose catastrophic risks, yet offer no convincing technical solutions.
About this quote
Quotesby Glyph