540: Uncensored AI on Linux

What this episode covers

We test two popular methods to run local language models on your Linux box. Then, we push the limits to see which language models will toe the line and which won't.Sponsored By:Tailscale: Tailscale is a programmable networking software that is private and secure by default - get it free on up to 100 devices!Linode Cloud Hosting: A special offer for all Linux Unplugged Podcast listeners and new Linode customers, visit linode.com/unplugged, and receive $100 towards your new account. Kolide: Kolide is a device trust solution for companies with Okta, and they ensure that if a device isn’t trusted and secure, it can’t log into your cloud apps.Support LINUX UnpluggedLinks:🎉 Alby — Boost into the show, first grab Alby, top it off, and then head over to the Podcast Index.⚡️ LINUX Unplugged on the Podcastindex.org — You can boost from the web. Once Alby is topped off, visit our page on the Podcast Index.Brent in Berlin Meetup — Brent is back in Berlin and we're having a LOW KEY meetup at the NixOS night at cbase.One Time Fiat Fun for SCaLE 2024Southern California Linux Expo 19Purchase a SCALE Pass 63First NixCon North America!NixCon North America | SCALE 21xNixCon North America Call for ProposalsTexas Linux Fest 2024LFNW 2024: Ready or notLFNW 2024: Call for SessionsJupiter Broadcasting Live Calendar2023 Unplugged TuxiesOllama — Get up and running with Llama 2 and other large language models locally.OllamaHub — Explore and Download Custom Ollama Modelfiles.ollama libraryOllama Manual InstallOllama Docker Imageollama-webuiLlamaGPTllama-gpt — A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!Continue.dev — An open-source autopilot in your IDERefact.ai — Refact is an open-source AI coding assistant with blazing-fast code completion, powerful code improvement tools, and chat.32-Bit Challenge Chat roomSpeech Note — Speech Note let you take, read and translate notes in multiple languages. It uses Speech to Text, Text to Speech and Machine Translation to do so. Text and voice processing take place entirely offline, locally on your computer, without using a network connection. Your privacy is always respected. No data is sent to the Internet.Speech Note on GitHubLFNW: Call for SponsorsFramework 13 (Intel) NixOS ModuleGeneBean's hyprland nix configRazer Core X - Thunderbolt™ 3 eGPUCompleteNoobs: Ubuntu KVM to Windows 10 - Thinkpad T470 - Razer Core X - GTX 970 -LINUX Unplugged 308: The One About GPU Passthrough — Our crew walks you through their PCI Passthrough setups that let them run Windows, macOS, and distro-hop all from one Linux machine.Pick: Marker — Convert PDF to markdown quickly with high accuracy