535: The You in the Machine

Adam Smith from UC Santa Cruz joins us to discuss local Small Language Models (SLMs) and building open, autonomous tools. We explore his BayLeaf AI counterplatform, transagency (the "me-and-the-car" concept of human-agent collaboration), context distillation, and running models locally without relying on data-harvesting corporate clouds. Adam also shares practical strategies for working with coding agents, including why plain-text AGENTS.md files outperform model weight fine-tuning for long-term knowledge retention. He also describes how he uses open-source tools like OpenWebUI, OpenCode and OpenChamber to integrate small and local LLM integration into development environments and web chats. Discussion Resources & Links Adam's Projects & Research: BayLeaf AI Counterplatform: Hyper-local, open-source AI infrastructure built for UC Santa Cruz. See also the BayLeaf Getting Started Guide and BayLeaf Blog. Adam Smith's UCSC Directory Page & Personal Website Adam Smith on YouTube & GitHub (@rndmcnlly) Open-Source Agent Harnesses & Wrappers: OpenCode – Open-source command-line coding agent harness with VS Code integration (comparable to Claude Code) OpenChamber – Graphical desktop wrapper for OpenCode with VS Code support (comparable to Claude Cowork) OpenWeb UI – Self-hosted web interface and desktop app for local and remote LLM models OpenRouter – Token broker/middleman routing requests across competitive LLM inference providers r/LocalLlama – Community subreddit for local LLMs, open weights, and hardware advice AGENTS.md Convention & Agent Skills Standard – Open standards for persistent context management Context Distillation Paper (arXiv) Adam's Guide to Building Transagent Capabilities: Scope by Project Roots: Harnesses like OpenCode and OpenChamber use project root folders to establish clear, safe boundaries for agent execution. Store Context in Plain Text: Keep global and project preferences in simple .md files (like AGENTS.md or GLOSSARY.md). Plain text files make your accumulated knowledge portable across different model front-ends and harnesses. Handle Errors by Rewinding: When your LLM or agent makes a mistake, don't yell at it or try to correct it inline—go back in time to before the error and alter your prompt or context to lead it away from the mistake. Refactor Preferences into Skills: As your AGENTS.md context grows, ask the agent to factor recurring preferences out into modular skill files (.agentskills). Teach Transagency: Distinguish between standalone agent capabilities (autonomous work) and transagent capabilities (how the agent works with you). Build Local Tools: Ask the agent to build desktop helpers (QuickLook triggers, push notifications, or fuzzy glossary lookup) to keep a shared orientation. Elecia: "Complete one project or start a dozen?" Adam: "Tens of thousands, and start hundreds more within the starting of the other projects recursively forever." Transcript