Text-to-Distribution Prediction with Quantile Tokens and Neighbor Context

This paper introduces Quantile Token Regression, a novel framework designed to improve how large language models predict full probability distributions from unstructured text. Unlike previous methods that rely on a single representation for all outputs, this approach inserts dedicated quantile tokens into the model’s input to create direct pathways for estimating specific distribution levels. The researchers further enhance accuracy by using retrieval-augmented grounding, which incorporates semantically similar "neighbor" examples and their known data patterns into the prompt. Their mathematical analysis demonstrates that using Wasserstein-based loss functions provides superior results over traditional pinball losses for this specific task. Extensive testing on Airbnb and Stack Overflow datasets proves that these techniques significantly reduce error rates and produce much sharper, more reliable predictions. Ultimately, the study offers a scalable architecture for complex tasks like price forecasting and risk assessment, where understanding uncertainty is as critical as predicting a central value.