Deep in the heart of data // Carl Steinbach // MLOps Coffee Sessions #22

Coffee Sessions #22 with Carl Steinbach of LinkedIn, Deep in the Heart of Data.Join the Community: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://go.mlops.community/YTJoinIn⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Get the newsletter: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://go.mlops.community/YTNewsletter⁠⁠⁠⁠⁠⁠//BioCarl is a Senior Staff Software Engineer and currently the Tech Lead for LinkedIn's Grid Development Team. He is a contributor to Emerging Architectures for Modern Data Infrastructure//Other links referenced by Carl:https://rise.cs.berkeley.edu/wp-content/uploads/2017/03/CIDR17.pdfhttps://www.youtube.com/watch?v=-xIai_FvcSk&ab_channel=WePayEngineeringhttps://softwareengineeringdaily.com/2019/10/23/linkedin-data-platform-with-carl-steinbach/https://www.slideshare.net/linkedin/carl-steinbach-open-sourcehttps://dreamsongs.com/RiseOfWorseIsBetter.htmlhttps://engineering.linkedin.com/blog/2017/03/a-checkup-with-dr--elephant--one-year-laterhttps://engineering.linkedin.com/https://engineering.linkedin.com/blog/2018/11/using-translatable-portable-UDFshttps://a16z.com/2020/10/15/the-emerging-architectures-for-modern-data-infrastructure/--------------- ✌️Connect With Us ✌️ -------------Join our Slack community: https://go.mlops.community/slackFollow us on Twitter: @mlopscommunitySign up for the next meetup: https://go.mlops.community/registerConnect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/Connect with David on LinkedIn: https://www.linkedin.com/in/aponteanalytics/Connect with Carl on LinkedIn: https://www.linkedin.com/in/carlsteinbach/Timestamps:[00:00] Introduction to Carl Steinbach[00:44] Carl's background[04:51] Breakdown of Transpiler[10:55] Advantages of Decoupling the Execution Layer[15:25] Differences between UDF (user-defined function) Functions and Views[18:45] How do you ensure the reproducibility of these Views?[23:58] Data structure evolution[27:55] Are Data Lakes and Data Warehouse fundamentally different things, or are they on a path towards conversion?[33:37] It's inevitable that people will start doing machine learning on databases[36:01] Who gets permission on what, especially when it comes to data and how sensitive things can be?[41:27] Security aspect of data  [43:40] Does it require a level of obstruction on top of the data of the file system?[45:48] Why do we go back and go forward, which sets this trend?