Fabric Dataflows Gen2: The Future of ETL in Microsoft Fabric and How to Migrate Your Existing Dataflows

Fabric Dataflows Gen2: The Future of ETL in Microsoft FabricIf you’ve been juggling separate Power Query scripts, Data Factory pipelines and custom SQL jobs just to keep your reports alive, Fabric Dataflows Gen2 is your chance to simplify everything. In this episode, I walk through why Microsoft rebuilt dataflows as a first‑class Fabric resource—so your transformations live once, feed Lakehouse, Warehouse and Power BI together, and stop breaking every time an upstream schema changesWe start with the pain of today’s fragmented ETL. You’ll recognize the pattern: the same logic duplicated across tools, parallel pipelines for different teams, and endless arguments about why numbers don’t match between reports that supposedly use the same source. I trace how this grew out of disconnected services and why it capped the scalability of even well‑run data teams.Then we dive into the core architecture of Dataflows Gen2. You’ll learn how compute separation lets you scale transformation power without inflating storage, how managed staging kills off zombie temp tables, and how the new authoring and deployment model finally treats ETL like a governed, versioned asset instead of a side script. With concrete examples, we look at how a single dataflow can support both detailed analytics and summarized reporting without cloning logic in three places.Finally, we explore what “seamless across Fabric” really means in practice. Instead of exporting and re‑importing data between tools, your dataflows feed Lakehouse, Warehouse and semantic models directly, so each part of Fabric sees the same, consistent transformation output. By the end, you’ll know when to move ETL into Dataflows Gen2, which legacy jobs to retire first, and how this shift can free up a significant chunk of the time your data team currently spends just keeping fragile pipelines alive.WHAT YOU’LL LEARNWhy fragmented ETL across Power Query, Data Factory and SQL has become a scaling bottleneck.How Fabric Dataflows Gen2 changes the architecture with compute separation and managed staging.How a single, governed dataflow can feed Lakehouse, Warehouse and Power BI without duplicated logic.When and how to start migrating existing ETL into Dataflows Gen2 for a more maintainable Fabric environment.THE CORE INSIGHTThe core insight of this episode is that ETL is no longer a side process glued onto your analytics stack—it’s a first‑class Fabric asset. Once your transformations live in Dataflows Gen2 instead of being scattered across scripts and pipelines, you get cleaner governance, easier scaling and far fewer surprises every time your data model changes.WHO THIS EPISODE IS FORData engineers and BI teams maintaining multiple overlapping ETL jobs in the Microsoft stack.Architects planning or running Microsoft Fabric who need a sustainable ETL strategy.Analytics leaders who want consistent, trustworthy numbers across reports without endless pipeline firefighting.ABOUT THE AUTHOR / HOSTMirko Peters is a Microsoft 365 and data platform consultant and host of the M365.FM podcast, helping organizations move from fragile, tool‑sprawled ETL to unified, Fabric‑first data architectures. He works with data and BI teams to design reusable Dataflows Gen2 patterns, deployment pipelines and governance models so transformation logic becomes an asset the whole organization can rely on, not a collection of hidden scripts.Become a supporter of this podcast: https://www.spreaker.com/podcast/m365-fm-modern-work-security-and-productivity-with-microsoft-365--6704921/support.