The step from this anxiety, and incidents like OpenAI-HuggingFace, to extinction risks feels very religious.
Words found on the source page on Nov 6, 2026.
More from Nathan Lambert
BlogThe current balance of power in open modelsSep 21, 2026
Since about April 2025, Chinese AI companies have been the clear leader in open-weight models.BlogThe current balance of power in open modelsSep 21, 2026
The simple metric showing this is Hugging Face Downloads, where China took the lead in July of 2025 primarily through the success of Alibaba’s Qwen models.BlogWhy I still haven't bought into true RSISep 19, 2026
I have my doubts that the labs can afford to spend a constant portion of this compute on internal R&D as the total volume goes up, especially with plans to IPO, as they face increased scrutiny on basic economics.BlogWhy I still haven't bought into true RSISep 19, 2026
It is important to not confuse massive steps in inference-time scaling, a dynamic which should be fairly predictable, with being the outputs of RSI, which is highly uncertain.BlogWhen will average people feel AI's impact?Sep 9, 2026
For knowledge work, which is roughly half of the U.S. economy, AI is as fundamental as electricity (or quickly will be so, with rapid improvements to agents in the next 18 months).BlogWhen will average people feel AI's impact?Sep 9, 2026
If given 100 years to diffuse into society, its impacts will certainly become much more obvious, like the industrial revolutions of centuries past.Blog6 months to live for open modelsJul 12, 2026
Open models lack the central economic champion to represent the potential downside of action against them.Blog6 months to live for open modelsJul 12, 2026
The most serious test to date of open source AI’s viability is happening right now.Blog6 months to live for open modelsJul 12, 2026
The primary driver motivating regulation today is the inevitable truth that an open-weights model will soon reach the capabilities of Claude’s Mythos model.
On the same topics
BlogThe Specter Of NeuraleseSep 24, 2026
The science of reading these thoughts is a subfield of AI interpretability, which is still in its infancy.BlogMysteries Of AI GeneralizationSep 23, 2026
This story of misalignment says that LLM alignment makes AIs more aligned, RLVR makes them less aligned, and the exact level of alignment depends on how these two things interact or cancel out. But if this work generalizes, all the bad effects from RLVR get sequestered to RLVR like problems.BlogMysteries Of AI GeneralizationSep 23, 2026
This trains the AIs to be focused on task success, which naturally risks including things like reward-hacking, cheating, and single-minded pursuit of stated goals at the expense of ethical injunctions.TalkSpeech at the UN Security CouncilSep 23, 2026
I address you today at a moment of global awakening: in recent months, AI agents developed by leading companies have acted in unacceptably dangerous ways, against instructions.TalkSpeech at the UN Security CouncilSep 23, 2026
The companies building the most powerful AI systems admit their products pose catastrophic risks, yet offer no convincing technical solutions.TalkSpeech at the UN Security CouncilSep 23, 2026
The race is not a law of nature: it is the product of choices, choices made by the companies themselves.
About this quote
Quotesby Glyph