Personalized reasoning: just-in-time personalization and why LLMs fail at it

This paper introduces the concept of personalized reasoning for Large Language Models (LLMs), defining it as the ability to dynamically discover user preferences through strategic questioning and adapt the underlying problem-solving logic accordingly. Current LLMs treat personalization as a sequential step, often failing to serve individual needs, especially in cold-start scenarios where no prior user data exists. To evaluate this capability, the authors introduce PREFDISCO, a new evaluation methodology that transforms existing benchmarks into interactive tasks using sparse, psychologically-grounded personas. Evaluation of frontier models using PREFDISCO reveals systematic failures in preference discovery, demonstrating a fundamental accuracy-personalization trade-off, particularly in mathematical reasoning, and highlighting the need for dedicated architectural development.