Four Seats: Where LLMs Actually Plug In

When Spotify's language model faithfully narrated the wrong number of artists discovered, it revealed a deeper problem: guardrails need to sit upstream of the model, not downstream. This episode maps the four specific places where large language models actually fit inside production recommender systems—and all four share identical problems: they're expensive at scale, and their output always requires external validation. Joey walks through the architecture, constraint by constraint, with research-backed choices and an honest accounting of what he hasn't built himself. Key timestamps: 00:00:00 - Spotify Wrapped and the confidently wrong bug 00:01:30 - Joey's honesty check: five years of backend work, zero production recsys experience 00:04:35 - Cold-start retrieval: why you don't throw away IDs 00:09:25 - Generative retrieval and what the vocabulary shift means 00:12:45 - The container is right; the contents might be wrong 00:14:20 - The fix revisited from Part 2 logic --- Sources & further reading: • Okura, Tagami, Ono, Tajima, *Embedding-based News Recommendation for Millions of • Singh et al., Better Generalization with Semantic IDs, arXiv:2306.08121, 2023-06-13 • Hou et al., Large Language Models are Zero-Shot Rankers for Recommender Systems • Sun et al., Is ChatGPT Good at Search? (RankGPT), arXiv:2304.09542, 2023, EMNLP 2023 • Rajput et al., Recommender Systems with Generative Retrieval (TIGER) • Yoshnee Raveendran, "Inside the Archive: The Tech Behind Your 2025 Wrapped • Anthropic, "Structured outputs" docs • Zheng et al., Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena • All URLs fetched and quoted from on 2026-10-15. • Users*, KDD 2017 — — IDs fail on expiring news: https://research.lycorp.co.jp/en/publications/233 • content vectors; CTR +23%, duration +10%. Abstract page only. • random IDs vs content features at YouTube ranking. Abstract.: https://arxiv.org/abs/2306.08121 • arXiv:2305.08845, 2023, ECIR 2024 — — position and popularity: https://arxiv.org/abs/2305.08845 • bias, bootstrapping, 200-user budget limit. Full PDF text. • permutation, sliding window, 10 calls per 100 passages, $556: https://arxiv.org/abs/2304.09542 • 440M distillation. arXiv HTML v3; its latency figures render garbled and are not used. • arXiv:2305.05065, 2023, NeurIPS 2023 — — Semantic IDs, cold: https://arxiv.org/abs/2305.05065 • start, invalid IDs, beam-search cost. Full PDF text. • Highlights", Spotify Engineering, 2026-03-12 • batch precompute: https://engineering.atspotify.com/2026/3/inside-the-archive-2025-wrapped • stats block, distillation, LLM judges on ~165,000 reports, the timezone bug. • [primary, vendor case study] Meta AI, "Spotify personalized recommendations built with Llama" • 2024-12-18: https://ai.meta.com/blog/spotify-personalized-recommendations-built-with-llama/ • LLM explanations; the unmethoded "four times" figure. • constrained decoding: https://platform.claude.com/docs/en/build-with-claude/structured-outputs • and its stated exceptions. • arXiv:2306.05685, 2023, NeurIPS 2023 D&B — — 80%+ agreement: https://arxiv.org/abs/2306.05685 • judge biases. Abstract. • [preprint] Wang et al., Large Language Models are not Fair Evaluators, arXiv:2305.17926 • 2023-05-29 — — order hacking, 66/80, calibration. Abstract.: https://arxiv.org/abs/2305.17926 This podcast episode was fully generated by AI — research, script, voices, and production. Built with Claude, Piper TTS, and automated pipeline tooling.