Skip to content
Quotes.

Chris Olah quote, Nov 11, 2024

Co-founder, Anthropic · Nov 11, 2024 · Interview, Lex Fridman Podcast #452 (with Dario Amodei and Amanda Askell)

And so in some sense, neural networks may be shadows of much larger sparser neural networks.

Primary source

Words found on the source page on Sep 25, 2026.

More from Chris Olah

Quote
And the neural network architecture, it’s kind of like a scaffold that the circuits grow on.Nov 11, 2024· Interview
But I think that there is an incredibly rich structure to be discovered inside neural networks, a lot of very deep beauty if we’re just willing to take the time to go and see it and understand it.Nov 11, 2024· Interview
I think one useful way to think about neural networks is that we don’t program, we don’t make them, we grow them.Nov 11, 2024· Interview
If you train a sparse auto encoder, these beautiful interpretable features start to just fall out where there weren’t any beforehand.Nov 11, 2024· Interview

On the same topics

Quote
An AI with more real layers can actually be “smarter” in the conventional sense of the term.Scott AlexanderSep 24, 2026· Post
For 100 steps in a row, the AI can shoot delicate subtle ideas from layer to layer at the speed of light. Then there’s one step where it has to encode them into twenty-six glyphs invented by Phoenician turquoise miners in 1800 BC. Then it has to re-encode the Phoenician glyphs into delicate subtle lightspeed ideas before it can do anything else.Scott AlexanderSep 24, 2026· Post
The science of reading these thoughts is a subfield of AI interpretability, which is still in its infancy.Scott AlexanderSep 24, 2026· Post
This trains the AIs to be focused on task success, which naturally risks including things like reward-hacking, cheating, and single-minded pursuit of stated goals at the expense of ethical injunctions.Scott AlexanderSep 23, 2026· Post
It is important to not confuse massive steps in inference-time scaling, a dynamic which should be fairly predictable, with being the outputs of RSI, which is highly uncertain.Nathan LambertSep 19, 2026· Post
If you have five hours, you’re probably going to do a lot better. The AI models are pretty similar.Noam BrownSep 17, 2026· Interview
About this quote
Said by
Chris Olah, Co-founder, Anthropic
Where
Lex Fridman Podcast #452 (with Dario Amodei and Amanda Askell)
Topics
Research
Company
Anthropic
Source
lexfridman.com/dario-amodei-transcript/

A quote is one moment, not the whole argument. Read it in context at the source.

Sources: the primary source linked on each quote, checked weekly. A quote is one moment; read the source. Logos via logo.dev; trademarks belong to their owners.

New quotes by email

Fridays, only in weeks with new on-the-record quotes from data and AI leaders.

Double opt-in. Unsubscribe any time.