How AI Agents Actually Work

Gartner: 40% of enterprise apps will include AI agents by end of 2026. And 40% of agent projects will be canceled by 2027. Same number, same organization, same year. This episode pulls apart the architecture under the marketing. A chatbot is a function — input in, output out, one round trip. An agent is a while loop. It keeps calling the model until the task is done. That's the entire distinction nobody gets right. Four components: LLM (brain), tools (hands), memory, planning loop. The loop runs Observe-Think-Act-Result and caps at 15-25 iterations because more iterations just burn tokens. Every iteration sends the entire conversation history again. The academic foundation: ReAct (Yao et al., Princeton + Google, Oct 2022). 34% improvement over reinforcement learning on ALFWorld and WebShop. A language model with a search tool beat systems specifically trained for those environments. The insight: not smarter models, models that can check their work. Tool calling: JSON schemas, plain-English descriptions, ~100 tools and 20 args per tool is the reliable ceiling. Vague descriptions cascade — a wrong tool at step three breaks every step after. Context management is the part nobody warns you about. Hong et al.: 87% of tokens in coding agents go to navigating the codebase, not writing it. Anthropic's Pokemon-playing agent kept a markdown file for long-term memory. Compaction, retrieval, sub-agents — the last one costs 7X more tokens. MCP: Anthropic open-sourced it November 2024. OpenAI March 2025. Google April. Linux Foundation handoff December. 10,000 active public servers, 97 million monthly SDK downloads. Devin: compound system, four models, 67% PR merge rate, contributing about a third of commits to a major open-source project. Enterprise reality: McKinsey 62% experimenting / 23% scaling. Gartner found ~130 real agent vendors out of thousands — ~95% agent washing. The reliability math that explains the cancellations: 95% per-step accuracy → 60% over 10 steps → 36% over 20. Token costs are 70% of agent OpEx; 60-80% wasted in retry loops. Three dated predictions for 2026-2028. RELATED EPISODES When AI Agents Go to Court — the Workday/Eightfold legal tributary on agentic hiring The Loop Closed in the Sandbox — agent autonomy and the capital-heavy/human-light frame Computer Use 45X — the operational layer once agent inference is cheap enough to loop Claude Mythos — the model behind the next generation of agent capability CHAPTERS 00:00 Cold open — Gartner's two 40% predictions 00:45 Chatbot vs agent — function vs while loop 01:46 The four components and the Observe-Think-Act-Result loop 03:06 ReAct (Princeton + Google, Oct 2022) — 34% over RL 04:31 How tool calling actually works — JSON schemas and the 100-tool ceiling 07:14 Context management — Hong et al's 87% navigation finding 08:17 Anthropic's Pokemon agent and the markdown-memory trick 09:51 MCP — from Anthropic standard to Linux Foundation in 14 months 12:24 Framework landscape — Agents SDK, LangGraph, CrewAI, AutoGen 13:57 Devin — compound 4-model system, 67% PR merge rate 15:11 Enterprise reality — McKinsey 62%/23%, Gartner's 130 real agents 17:00 The reliability math — 95% per step → 36% over 20 steps 17:58 OWASP Top 10 for agents and alignment under stress 19:13 The cost math — 70% OpEx, 60-80% wasted tokens, model routing 20:38 When to use an agent — and when not to 23:12 Three predictions for 2026-2028 and the closing tension SOURCES Gartner — agentic AI cancellation + adoption forecasts (2025-2026) Yao et al. — ReAct: Reasoning + Acting (Princeton/Google, Oct 2022) Hong et al. — context-attention degradation in coding agents Anthropic — MCP open standard (Nov 2024) + Linux Foundation handoff McKinsey — 2025 State of AI Cognition Labs — Devin benchmarks + PR merge rates OWASP — Top 10 for LLM and Agentic Applications Stack Overflow — 2026 developer survey on agents