Quotesby GlyphResearch quotes
88 quotes from 36 people
- An AI with more real layers can actually be “smarter” in the conventional sense of the term.
+2more from this blog
- For 100 steps in a row, the AI can shoot delicate subtle ideas from layer to layer at the speed of light. Then there’s one step where it has to encode them into twenty-six glyphs invented by Phoenician turquoise miners in 1800 BC. Then it has to re-encode the Phoenician glyphs into delicate subtle lightspeed ideas before it can do anything else.
- The science of reading these thoughts is a subfield of AI interpretability, which is still in its infancy.
- Every quote from Scott Alexander
- This trains the AIs to be focused on task success, which naturally risks including things like reward-hacking, cheating, and single-minded pursuit of stated goals at the expense of ethical injunctions.
- It is important to not confuse massive steps in inference-time scaling, a dynamic which should be fairly predictable, with being the outputs of RSI, which is highly uncertain.
- If you have five hours, you’re probably going to do a lot better. The AI models are pretty similar.
+2more from this podcast
- So I was like, “I don’t think we’re going to get it in 2026, probably not in 2027, maybe in 2028.” So it did happen a lot faster than I expected.
- The concerning scenario is that, especially as these models are becoming more capable, we make them what we think is aligned, and they’re 99.9% aligned. Then we use these models to help us with the next generation of models, and they end up being 99.8% aligned. Then with each subsequent generation, we see an increasing degradation in alignment.
- Every quote from Noam Brown
- If I was in charge of an alignment or interpretability team I would never run out of experiments to run and things to try, including having the time to properly talk to the models in the first place.
+1more from this blog
- The proof AI produced runs to more than 500 pages and will not be intelligible to any human. That is a strange milestone in our history, in philosophy, in science and in mathematics that could fundamentally change our relationship with knowledge – humans won’t be able to inspect the proof, or understand it at all.
- Bottom line: these hypotheses suggest that as AI capabilities keep growing, this kind of behavior could keep growing in severity too, unless we revisit the principles by which the most advanced models are trained.
- Basically anything that can be learned through RL can be distilled very easily, because it’s a small number of bits.
+3more from this podcast
- In fact, I remember in the early OpenAI days having the intuition that just minimizing log loss wasn’t going to get you to intelligence.
- Right now, you don’t get explosive growth in capabilities because you still get bottlenecked enough when you’re trying to do research and engineering.
- So I would expect that we’re nowhere near the ceiling of how well you can do research.
- Every quote from John Schulman
- My first concern is that, since roughly this summer, AI has been advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI.
- But a secondary revolution will come from using AI as a microscope: we will use it to see our own minds.
+1more from this blog
- I don’t think you can have AIs that perform whole jobs as competently as humans if they’re forced to just write Markdown files from session to session.
+1more from this essay
- Meanwhile, there is structural uncertainty at the early stages of every big new technology, but the uncertainty now is different, because we don’t have a good theoretical understanding of why these models work so well and so we don’t know how much better they can get.
- This significant increase in the automation of coding has been equivalent to dropping many, many more employees into Anthropic, speeding up our overall pace of development.
+1more from this talk
- Gemini Omni doesn't just build scenes that look real, it reasons about what should happen next.
+1more from this blog
- The top AI researchers became post-economic all at once. (Maybe the closest prior analogues is the early crypto HODLRs?).
- The problem in this field is too much opportunity. The thing that we observe very early on in OpenAI is that everything we could imagine works now.
- Whereas in the past, you may need to sit and envision five years out. The models are going to be dramatically different in a year’s time.
- The speed of advancement in AI demands a different approach to how we build, how we organize, and where we focus. Labs gives us room to break the mold and explore...
- This feedback loop is gathering steam month by month, and may be only 1–2 years away from a point where the current generation of AI autonomously builds the next.
- Training on the test set is a new art form. What does it look like to crush all the benchmarks but still not get AGI?
+1more from this blog
- Anthropic was founded by scientists who believe AI can deliver transformative progress for research itself...
- All the pre-trained models are pretty much the same because they pre-train on the same data.
+6more from this podcast
- But now that compute is big, compute is now very big, in some sense we are back to the age of research.
- I think that’s been guiding me a fair bit, thinking from multiple angles and looking for almost beauty, beauty and simplicity.
- One consequence of the age of scaling is that scaling sucked out all the air in the room. Because scaling sucked out all the air in the room, everyone started to do the same thing.
- So for research, you definitely need some amount of compute, but it’s far from obvious that you need the absolutely largest amount of compute ever for research.
- So it’s back to the age of research again, just with big computers.
- The thing which I think is the most fundamental is that these models somehow just generalize dramatically worse than people.
- Every quote from Ilya Sutskever
- It’s plausible that in about five years, there will be quantum computers capable of breaking the RSA and ECC encryption that secures the vast majority of internet communications, financial transactions, and your sensitive personal data.
- I would’ve not thought, none of us thought that somehow this AI thing and a deep neural network at some scaling will suddenly become the thing that figures out the patterns, not some schematized data model.
- The more a task/job is verifiable, the more amenable it is to automation in the new programming paradigm.
- Building spatially intelligent AI requires something even more ambitious than LLMs: world models, a new type of generative models whose capabilities of understanding, reasoning, generation and interaction with the semantically, physically, geometrically and dynamically complex worlds - virtual or real - are far beyond the reach of today’s LLMs.
+2more from this essay
- If you talk to a billion parameter model, I think in 20 years, you can have a very productive conversation.
+3more from this podcast
- Reinforcement learning is a lot worse than I think the average person thinks.
- The way I like to put it is you’re sucking supervision through a straw.
- We’re building ghosts or spirits or whatever people want to call it, because we’re not doing training by evolution. We’re doing training by imitation of humans and the data that they’ve put on the Internet.
- Every quote from Andrej Karpathy
- In which case, it will be another instance of the bitter lesson, that the things that used human knowledge were eventually superseded by things that just trained from experience and computation.
+2more from this podcast
- Large language models are trying to get by without having a goal or a sense of better or worse. That’s just exactly starting in the wrong place.
- What we want, to quote Alan Turing, is a machine that can learn from experience, where experience is the things that actually happen in your life.
- Every quote from Richard Sutton
- If we can do a decade’s worth of research in a year, or a month, then the rate of progress will obviously be quite different.
+2more from this blog
- I think actually for fundamental original research, universities are better than the big companies.
- As you start getting more intelligence in one part of the stack, you’re just going to run into a different set of bottlenecks. That’s how engineering always works: solve one bottleneck, you get another bottleneck.
- So the crucial turning point is mid-2027, when they’ve basically fully automated the AI R&D process and they’ve got this corporation within a corporation, the army of geniuses that are autonomously doing all this research and they’re continually being trained to improve their skills, blah, blah, blah.
- Anthropic is doubling down on interpretability, and we have a goal of getting to “interpretability can reliably detect most model problems” by 2027.
+2more from this essay
- This is why we like the analogy of thinking of this as the transistor moment of quantum computing, where we effectively have a new phase, which is the topological phase, which means we can even now reliably hide the quantum information, measure it, and we can fabricate it.
- The intelligence of an AI model roughly equals the log of the resources used to train and run it.
- Passing ARC-AGI does not equate to achieving AGI, and, as a matter of fact, I don't think o3 is AGI yet. o3 still fails on some very easy tasks, indicating fundamental differences with human intelligence.
+1more from this blog
- This new form of AI excels at modeling human intuition rather than human reasoning...
+1more from this talk
- And so in some sense, neural networks may be shadows of much larger sparser neural networks.
+4more from this podcast
- And the neural network architecture, it’s kind of like a scaffold that the circuits grow on.
- But I think that there is an incredibly rich structure to be discovered inside neural networks, a lot of very deep beauty if we’re just willing to take the time to go and see it and understand it.
- I think one useful way to think about neural networks is that we don’t program, we don’t make them, we grow them.
- If you train a sparse auto encoder, these beautiful interpretable features start to just fall out where there weren’t any beforehand.
- Every quote from Chris Olah
- I’m pretty skeptical that we’re going to see an LLM do 80% in a year.
+3more from this podcast
- Now LLMs have sucked the oxygen out of the room. Everyone is just doing LLMs. I see LLMs as more of an off-ramp on the path to AGI actually.
- OpenAI basically set back progress towards AGI by quite a few years, probably like 5-10 years.
- The way LLMs work is that they’re basically this big interpolative memory.
- Every quote from Francois Chollet
- Hundreds of millions of AGIs could automate AI research, compressing a decade of algorithmic progress (5+ OOMs) into ≤1 year.
- I make the following claim: it is strikingly plausible that by 2027, models will be able to do the work of an AI researcher/engineer.
- So until we have, again, world models, systems that can train themselves to understand how the world works, we’re not going to have significant progress in robotics.
- In terms of your question of large models, I think we’ve got to push scaling as hard as we can and that’s what we’re doing here.
+2more from this podcast
- It’s interesting to me that large language models in their current form are not inventions, they’re discoveries. The telescope was an invention, but looking through it at Jupiter, knowing that it had moons, was a discovery.
+1more from this podcast
- Last year, we anticipated it would be nearly a decade before we had intern-level code generation, Hollywood-quality videos or human quality speech that didn’t sound mechanical. But a quick listen to Eleven Labs’ voices on TikTok or Runway’s AI film festival makes it clear that the future has arrived at warp speed.
- Foundation Models are “ few shot learners ”, exhibiting in-context learning and even zero shot transfer capabilities that generalize beyond the original intent of model trainers. In other words, the people creating the models don’t fully know what they are capable of.
