Future of the Data Engineering Role

Twenty years of new tools were supposed to make data engineering easier. Shachar Meir (former Director of Data Engineering at Meta, former data lead at PayPal) argues they made it more chaotic. He explains how "store now, model later" broke the data contract we're now scrambling to rebuild, why data teams fail for reasons that have nothing to do with technology, and the two tips he gives every CDO. Plus the photographer analogy that explains why he's still not convinced data engineering is going anywhere. Hosted by Ido Bronstein on Data Splash.⏱ CHAPTERS 0:00 Introduction 0:59 The 30 Second Splash 1:34 What a data advisor actually does 2:50 Why data teams fail: the missing ingredients 4:02 The DBA era — when data was always modeled 7:03 Breaking the contract: Hadoop and data lakes 8:45 Did technology make the job easier? 9:46 Two tips for every CDO 11:34 Enter AI: risks and the how-vs-what shift 15:03 Shifting ownership to the business 16:46 Is managing data still a profession? 18:46 The photographer analogy 19:06 Career advice for early-career data people 21:34 Why the business matters more than ever 22:28 Wrap-up🎙 ABOUT DATA SPLASH: Data Splash is a podcast for data engineers, data leaders, and anyone trying to make sense of AI and data right now. Brought to you by Upriver.🔔 Subscribe for new episodes weekly.🔗 LINKS • Upriver: [https://www.upriverdata.com/] • Connect with Shachar Meir: [https://www.linkedin.com/in/shacharmeir/] • Connect with Ido Bronstein: [https://www.linkedin.com/in/ido-bronstein/]#DataEngineering #AI #DataPlatform #LLMs #AIAgents