You can't Google your way to a recommender system

Schaun Wheeler and DJ Rich delve into the intricacies of building recommender systems in this podcast hosted by Arpit Choudhury. The discussion highlights the steps to developing a recommender system, practical advice for startups, and the evolving landscape of recommender technologies. KEY POINTS: * Identify the Problem: Recommender systems address the "discovery problem" by helping users sift through vast amounts of options to find relevant content quickly. Recognizing this problem is crucial before diving into solutions.* System Components: Recommender systems are complex and involve multiple components such as:* Item Inventory: Detailed metadata about items (e.g., descriptions, categories).* User Interaction History: Data on user interactions with items (e.g., views, purchases).* Recommendation Model: The core model that filters and ranks items based on user preferences.* The Learner: An important component, which trains the model and separates it from the model's deployment phase.* Build vs. Buy: Should one build a recommender system from scratch or use existing solutions? For many startups, buying an off-the-shelf system can be more practical due to advances in data infrastructure and the complexity of developing a bespoke system. Buying a system can also free up resources for other critical areas.* Practical Recommendations for Startups: Instead of getting bogged down by complex models initially, startups are encouraged to start with simpler models and leverage existing infrastructure to implement a functional recommender system.* Innovations in Recommender Systems: Schaun is interested in combining traditional methods with reinforcement learning to enhance system performance. DJ is excited about research addressing causal questions and handling sequential recommendations.REFERENCES:* "Are we really making progress?" This paper is a replication study on recommender algorithms and shows that many DL approaches couldn't be reproduced or could be beaten with linear methods. * "Deep Exploration for Recommender Systems" This paper talks about sequential decisioning for RSs (where you consider more than just one item recommendation). * "Two Decades of Recommender Systems at Amazon" This paper is a retrospective on what's work well at Amazon. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit edge.aampe.com