RECOMMENDER SYSTEM: Why They Update Models 100 Times a Day // Gleb Abroskin // MLOps Coffee Sessions #123

MLOps Coffee Sessions #123 with Gleb Abroskin, Machine Learning Engineer at Funcorp, RECOMMENDER SYSTEM: Why They Update Models 100 Times a Day, co-hosted by Jake Noble.Join the Community: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://go.mlops.community/YTJoinIn⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Get the newsletter: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://go.mlops.community/YTNewsletter⁠⁠⁠// AbstractFunCorp was a top 10 app store. It was a very popular app that had a ton of downloads and just memes. They need a recommendation system on top of that. Memes are super tricky because they're user-generated and they evolve very quickly. They're going to live and die by the Recommender System in that product.It's incredible to see FunCorp's maturity. Gleb breaks down the feature store they created and the velocity they have to be able to create a whole new pipeline in a new model and put it into production after only a month!// BioGleb makes models go brrrrr. He doesn't know what is expected in this field, to be honest, but Gleb has experience in deploying a lot of different ML models for CV, speech recognition, and RecSys in a variety of languages (C++, Python, Kotlin), serving millions of users worldwide./ MLOps Jobs board  jobs.mlops.communityMLOps Swag/Merchhttps://mlops-community.myshopify.com/// Related LinksPutting a two-layered recommendation system into production - https://medium.com/@FunCorp/putting-a-two-layered-recommendation-system-into-production-b8caaf61393d Practical Guide to Create a Two-Layered Recommendation System - https://medium.com/@FunCorp/practical-guide-to-create-a-two-layered-recommendation-system-5486b42f9f63 Ten Mistakes to Avoid When Creating a Recommendation System - https://medium.com/@FunCorp/ten-mistakes-to-avoid-when-creating-a-recommendation-system-8268ed60aeba Applying Domain-Driven Design and Patterns: With Examples in C# and .NET, 1st Edition by Jimmy Nilsson: https://www.amazon.com/Applying-Domain-Driven-Design-Patterns-Examples/dp/0321268202--------------- ✌️Connect With Us ✌️ -------------Join our Slack community: https://go.mlops.community/slackFollow us on Twitter: @mlopscommunitySign up for the next meetup: https://go.mlops.community/registerCatch all episodes, blogs, newsletters, and more: https://mlops.community/Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/Connect with Jake on LinkedIn: https://www.linkedin.com/in/jakednoble/Connect with Gleb on LinkedIn: https://www.linkedin.com/in/gasabr/Timestamps:[00:00] Introduction to Gleb Abroskin[00:50] Takeaways[05:39] Breakdown of FunCorp teams[06:47] FunCorp's team ratio[07:41] FunCorp team provisions[08:48] Feature Store vision[10:16] Matrix factorization[11:51] Fairly modular, fairly thin infrastructure[12:26] Distinct models with the same feature[13:08] FunCorp's definition of Feature Store[15:10] Unified API[15:55] FunCorp's scaling direction[17:01] Level up as needed[17:38] Future of FunCorp's Feature Store[18:37] Monitoring investment in the space[19:43] Latency for business metrics[21:04] Velocity to production[23:10] 30-day retention struggle[24:45] Back-end business stability[27:49] Recommender systems[30:34] Back-end layer headaches[37:41] Continuous training pipelines produce an artifact[39:33] Worst-case scenario[40:38] Realistic estimation of a new model deployment[41:42] Recommender Systems' future velocity  [43:07] A/B Testing launch - no launch decision[46:32] Lightning question[47:08] Wrap up