Higher stakes: with access to tools, agents are capable of performing more impactful tasks, but any failure could have more severe consequences.
Words found on the source page on Nov 6, 2026.
More from Chip Huyen
BlogCommon pitfalls when building generative AI applicationsJan 16, 2025
The teams with the best products I’ve seen all have human evaluation to supplement their automated evaluation.BlogCommon pitfalls when building generative AI applicationsJan 16, 2025
These annotation guidelines can help improve the model’s instructions (if a human has a hard time following the instructions, the model will, too).BlogAgentsJan 7, 2025
A common complaint is that agents are only good for burning through your API credits. However, if agents can be autonomous, they can save human time, making their costs worthwhile.BlogAgentsJan 7, 2025
Compound mistakes: an agent often needs to perform multiple steps to accomplish a task, and the overall accuracy decreases as the number of steps increases. If the model’s accuracy is 95% per step, over 10 steps, the accuracy will drop to 60%, and over 100 steps, the accuracy will be only 0.6%.
On the same topics
BlogWill agents have network effects?Oct 6, 2026
If the content creation loop works for agentic virality, I think it means there’s a big incentive for every agent to volunteer to make visually-appealing, shareable content, do this often.UnverifiedBlogThe Specter Of NeuraleseSep 24, 2026
The science of reading these thoughts is a subfield of AI interpretability, which is still in its infancy.BlogThinking in Systems, Shipping in LoopsSep 24, 2026
This is the new software engineering: designing the systems that let AI write code correctly at scale.BlogMysteries Of AI GeneralizationSep 23, 2026
This story of misalignment says that LLM alignment makes AIs more aligned, RLVR makes them less aligned, and the exact level of alignment depends on how these two things interact or cancel out. But if this work generalizes, all the bad effects from RLVR get sequestered to RLVR like problems.BlogMysteries Of AI GeneralizationSep 23, 2026
This trains the AIs to be focused on task success, which naturally risks including things like reward-hacking, cheating, and single-minded pursuit of stated goals at the expense of ethical injunctions.TalkSpeech at the UN Security CouncilSep 23, 2026
I address you today at a moment of global awakening: in recent months, AI agents developed by leading companies have acted in unacceptably dangerous ways, against instructions.
About this quote
Quotesby Glyph