Self-improving LLM agents at Test-Time

This is research paper introduces and evaluates a novel framework called Test-Time Self-Improvement (TT-SI) for large language model (LLM) agents. This approach focuses on improving model performance efficiently during inference by adapting to challenging examples on the fly. The method involves three key steps: Self-Awareness (identifying uncertain test inputs), Self-Data Augmentation (generating similar training examples from these uncertain inputs), and Self-Improvement (performing a lightweight fine-tuning on the generated data). Empirical results across multiple agent benchmarks demonstrate that TT-SI significantly improves accuracy compared to a base model, often requiring 68 times less training data than traditional supervised fine-tuning. A graphical figure and tables illustrate the framework and quantify the substantial accuracy gains achieved by the TT-SI and its variant, Test-Time Distillation (TT-D), particularly when adapting to a single generated sample per uncertain case. The authors propose that this methodology offers a more cost-effective and generalizable paradigm for building capable, self-evolving LLM agents.