Self-improving LLM agents at test-time

The academic paper proposes a novel framework called Test-Time Self-Improvement (TT-SI) for training Large Language Model (LLM) agents more efficiently by adapting them on-the-fly during inference. This new paradigm is motivated by the high cost and inefficiency of traditional large-scale fine-tuning, which often involves redundant data. TT-SI operates in three steps: Self-Awareness identifies uncertain test instances, Self-Augmentation generates tailored training samples for those instances, and Self-Improvement uses these samples for lightweight, temporary fine-tuning. Empirical results across several agent benchmarks demonstrate that TT-SI significantly improves model accuracy (e.g., +5.48% on average) while utilizing dramatically fewer training samples compared to standard supervised fine-tuning. The findings support the potential of uncertainty-guided, instance-specific learning as a more effective and cost-efficient approach for building capable, self-evolving LLM agents.