EP 38: The Local AI Stack Nobody Talks About (But Should)

You want to run AI locally. You have questions: What hardware do I actually need? Which framework should I use? How much will this cost? What's the realistic performance? In this episode, Sam brings back Trent Rossiter, founder of Logical Data Solutions, for a practical walkthrough of building a production-grade local AI lab. Trent has built real systems for enterprise clients, tested frameworks on multiple hardware stacks, and made the hardware choices that matter. This is not theory. This is what actually works. WHAT WE COVER: ▪  Hardware & Framework Choices: VRAM is the critical metric (not all VRAM is equal — memory throughput matters as much as capacity).  ▪  Model Architecture & Capability: Mixture of Experts (MoE) lets you fit more power into less VRAM by using fewer active parameters.  ▪  Real Enterprise Applications: Computer vision for quality assurance on assembly lines. Proprietary data handling without cloud exposure.  ▪  Your Starter Stack (All Free): Langflow (agentic workflow builder), Goose (MCP-enabled chat), AnythingLLM (with vector stores for RAG), MCP servers (Model Context Protocol — standardised tool integration).  ▪  Agentic AI & Security: OpenClaw is powerful but controversial — manages email, Telegram, calendars, creates sub-agents. Trent runs it in Docker on an isolated machine for safety. NVIDIA's NemoClaw is the enterprise version (security-first, nothing-allowed-by-default, explicit permissions). HARDWARE TRENT MENTIONS: NVIDIA DGX Spark — 128GB unified memory, CUDA stack Apple MacBook Pro/Mac mini — up to 512GB unified memory, market leader for personal AI AMD integrated AI PCs — emerging competitor NVIDIA RTX gaming cards (30/40/50/60 series) — high VRAM, high power consumption, complex FIND TRENT ROSSITER: LinkedIn: https://www.linkedin.com/in/benjamin-trent-rossiter-mba-0157945/ Logic Data Solutions: https://logicdatasolutions.com/ Contact: [email protected]