Fabric Notebooks: When Interactive Spark Meets Production Reality

Fabric Notebooks: When Interactive Spark Meets Production Reality Episode 3 • 2026-01-16 Duration: 11:26 Matthias and Fabia dig into Fabric Notebooks — the code-first data engineering workhorse. They cover starter pools vs. custom environments, the new native Python mode, the notebook-to-production gap, and why your notebook that runs perfectly at 2pm crashes in the pipeline at 2am. What we discuss A real-world mistake from a pre-Fabric era The one question that reframes the architectural debate How we got here — predecessor products and evolution Why the "obvious" answer is often wrong A real Reddit/Microsoft Q&A question unpacked The concrete recommended architecture F-SKU realism — what this actually costs When the rejected approach is actually right Risks of the recommended path What Microsoft is shipping that changes the calculus The architectural principle to take home Key takeaways Completely fair. And I'll go further — I've seen teams spin up Spark to process ten megabytes of CSV. That's like renting a crane to hang a picture frame. If your transformation is row-level, SQL-expressible, and under a gig? A warehouse... Nope. It's one or the other. And your team will absolutely discover this at the worst possible moment — usually during a demo. The play is: develop and explore on starter pools, switch to a custom Environment only for final testing and... And teams do exactly that. Until they need great_expectations, or dbt-core, or some internal package. Starter pools give you the default library set — pandas, scikit-learn, plotly, the usual. But the moment you need anything custom, you... Resources What is a notebook? Python experience in notebooks Configure Starter Pools Lakehouse and notebooks Data Wrangler Manage libraries Schedule notebook High Concurrency mode VS Code extension Spark Job Definition Notebook best practices Author and execute notebooks Notebook visualization About the show Built on ElevenLabs voice synthesis. Matthias — cloned voice. Fabia — designed AI co-host. See Matthias live on YouTube (Fabric Friday), at his meetups, and at conferences like FabCon. Hosted by Matthias Falland — Microsoft Data Platform MVP and community architect behind the Fabric Periodic Table. New episodes every Friday. Submit your case Have an architecture decision you are wrestling with? DM Matthias on LinkedIn — find him as Matthias Falland. Three to five sentences about the decision, your team size, and your current stack. We anonymize before airing. Built on ElevenLabs voice synthesis. Brand design based on fabricperiodictable.com.