024 - On-Prem AI with Ollama

This week on SysAdmin Weekly, Andy is joined once again by Mike Nelson for a deep dive into the world of local AI. Together, they unpack Ollama and Open WebUI, exploring how SysAdmins can run large language models on-prem to unlock powerful workflows while keeping sensitive data in-house.From hardware setups (yes, even that dusty old GPU in your basement), to why privacy-conscious SysAdmins are moving workloads local, Andy and Mike cover it all. Along the way, they swap stories on home labs, Ubiquiti gear, Ghost CMS, Hugging Face models, and where open-source AI really shines.If you’ve ever wondered why you’d bother with local AI when ChatGPT is just a browser tab away, this episode lays out the practical sysadmin use cases, security considerations, and future of AI in the trenches.Episode Resources:- SysAdmin Weekly Companion Newsletter- AndyOnTech- Project Runspace- Ollama Models Directory- BookStack Wiki Software- Confluence Free Instance (Atlassian)- Hugging Face Model Hub- Awesome MCP Servers (GitHub curated list)- Previous SysAdmin Weekly Episode: “The Importance of IT Documentation”- Previous SysAdmin Weekly Episode: “Building a Home Lab on the Cheap”