Speed Beat Relevance: What Broke When I Put an LLM in Front of Product Search

This story was originally published on HackerNoon at: https://hackernoon.com/speed-beat-relevance-what-broke-when-i-put-an-llm-in-front-of-product-search. Six failures from building an LLM-backed product search engine: a price filter that never filtered, invented category IDs, and latency that beat relevance. Check more stories related to machine-learning at: https://hackernoon.com/c/machine-learning. You can also check exclusive content about #ai, #search, #ecommerce, #llm, #engineering, #product-search, #software-engineering, #lessons-learned, and more. This story was written by: @ohadfarkash. Learn more about this writer by checking @ohadfarkash's about page, and for more stories, please visit hackernoon.com. I built a natural-language product search engine over a marketplace catalogue in twelve languages. The interesting failures were not in the model's language understanding — that part mostly worked. They were in the seams: a price filter that had never once filtered, an LLM confidently inventing valid-looking category IDs, a substring match that turned "newborn" into "born" for years, and the finding that a six-second cold search lost more shoppers than an imperfect result ever did.