Real-time Feature Pipelines, A Personal History // Hendrik Brackmann // MLOps Meetup #46

MLOps community meetup #46! Last Wednesday, we talked to Hendrik Brackmann, Director of Data Science and Analytics at Tide.Join the Community: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://go.mlops.community/YTJoinIn⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Get the newsletter: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://go.mlops.community/YTNewsletter⁠⁠// Abstract:Tide is a U.K.-based FinTech startup with offices in London, Sofia, and Hyderabad. It is one of the first and the largest business banking platforms in the UK, with over 150,000 SME members. As of 2019, one of Tide’s main focuses is to be data-driven. This resulted in the formation of a Data Science and Analytics Team with Hendrik Brackmann at its head. Let's witness Hendrik's personal anecdotes in this episode!// Bio:After studying probability theory at the University of Oxford, Hendrik joined MarketFinance, an SME lender, in order to develop their risk models. Following multiple years of learning, he joined Finiata, a Polish and German lender, in order to build out their data science function. Not only did he succeed in improving the risk metrics of the company, but he also learnt to manage a different department as interim Head of Marketing.Hendrik's job as Director of Data Science and Analytics at business bank Tide is to oversee data engineering, data science, insights and analytics, and data governance functions of Tide.// Final thoughtsPlease feel free to drop some questions you may have beforehand into our Slack channel(https://go.mlops.community/slack)Watch some old meetups on our YouTube channel:https://www.youtube.com/channel/UCG6qpjVnBTTT8wLGBygANOQ----------- Connect With Us ✌️-------------   Join our Slack community:  https://go.mlops.community/slackFollow us on Twitter:  @mlopscommunitySign up for the next meetup: https://go.mlops.community/registerConnect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/Connect with Hendrik on LinkedIn: https://www.linkedin.com/in/hendrik-brackmann-b2b5477a/Timestamps:[00:00] Introduction to Hendrik Brackmann [01:54] Hendrik's background in tech [03:22] First Phase of the three epic journeys of Hendrik [08:05] Were there some hiccups you were running into as you're trying to make things better?  [10:50] Any other learnings that you got from that job that you want to pass along to us? [11:50] You were doing all batches at that point, right? [12:35] Phase 2: of Hendrik's epic journey [15:11] Did you eventually cut down at the time that it took? [15:50] Breakdown of Transformation terminologies and their importance [19:03] What are some things that you would never do again? [20:32] How did you see things more clearly? [22:30] Phase 3: Moving on to Tide [24:46] Have you only worked with teams with one programming language?[30:47] Did you try to open-source solutions, or did you just go right out to buy it? [33:12] What is real-time for you? How much latency is there? How much time do you need?[37:18] At what stage did you realize to get the feature store?  [40:09] What would you recommend from a maturity standpoint to get a feature store? [41:20] Can you summarize some of the greatest problems that the feature stores solve for you?   [42:22] What problems does a feature store introduce, if any? [44:39] Where do the model and the feature start from the perspective of a system in engineering?[49:15] You need good data management in feature stores[50:21] Have you ever used or built any feature stores that explicitly handle units and do dimensional analysis on derived features? [54:46] What kind of models do you have up at the moment, and how do you test and monitor, and deploy the models?