Home Speakers Ilya Sutskever Quote All the pre-trained models are pretty much the same because they pre-train on the same data.
Words found on the source page on Nov 6, 2026.
More from Ilya Sutskever Podcast Dwarkesh Podcast: We're moving from the age of scaling to the age of research Nov 25, 2025
But now that compute is big, compute is now very big, in some sense we are back to the age of research. Podcast Dwarkesh Podcast: We're moving from the age of scaling to the age of research Nov 25, 2025
I think that’s been guiding me a fair bit, thinking from multiple angles and looking for almost beauty, beauty and simplicity. Podcast Dwarkesh Podcast: We're moving from the age of scaling to the age of research Nov 25, 2025
If you think about the term “AGI”, especially in the context of pre-training, you will realize that a human being is not an AGI. Podcast Dwarkesh Podcast: We're moving from the age of scaling to the age of research Nov 25, 2025
Number three, I think it would be really materially helpful if the power of the most powerful superintelligence was somehow capped because it would address a lot of these concerns. Podcast Dwarkesh Podcast: We're moving from the age of scaling to the age of research Nov 25, 2025
One consequence of the age of scaling is that scaling sucked out all the air in the room. Because scaling sucked out all the air in the room, everyone started to do the same thing. Podcast Dwarkesh Podcast: We're moving from the age of scaling to the age of research Nov 25, 2025
So for research, you definitely need some amount of compute, but it’s far from obvious that you need the absolutely largest amount of compute ever for research. Podcast Dwarkesh Podcast: We're moving from the age of scaling to the age of research Nov 25, 2025
So it’s back to the age of research again, just with big computers. Podcast Dwarkesh Podcast: We're moving from the age of scaling to the age of research Nov 25, 2025
The thing which I think is the most fundamental is that these models somehow just generalize dramatically worse than people.
On the same topics Blog The Specter Of Neuralese Sep 24, 2026
An AI with more real layers can actually be “smarter” in the conventional sense of the term. Blog The Specter Of Neuralese Sep 24, 2026
For 100 steps in a row, the AI can shoot delicate subtle ideas from layer to layer at the speed of light. Then there’s one step where it has to encode them into twenty-six glyphs invented by Phoenician turquoise miners in 1800 BC. Then it has to re-encode the Phoenician glyphs into delicate subtle lightspeed ideas before it can do anything else. Blog The Specter Of Neuralese Sep 24, 2026
The science of reading these thoughts is a subfield of AI interpretability, which is still in its infancy. Blog Mysteries Of AI Generalization Sep 23, 2026
This trains the AIs to be focused on task success, which naturally risks including things like reward-hacking, cheating, and single-minded pursuit of stated goals at the expense of ethical injunctions. Blog Why I still haven't bought into true RSI Sep 19, 2026
It is important to not confuse massive steps in inference-time scaling, a dynamic which should be fairly predictable, with being the outputs of RSI, which is highly uncertain. Podcast Dwarkesh Podcast: Agent swarms, alignment, and recursive self-improvement Sep 17, 2026
If you have five hours, you’re probably going to do a lot better. The AI models are pretty similar. About this quote 
Quotesby Glyph