metaTextGrad: Learning to learn with language models as optimizers

This academic paper introduces metaTextGrad, a novel meta-learning approach designed to enhance large language model (LLM) performance during inference by learning better loss functions and initialization strategies, referred to as inference templates. While existing methods like TextGrad refine LLM outputs iteratively, they often require extensive manual tuning and are sensitive to prompt wording. metaTextGrad addresses these limitations by employing a meta-learning framework that optimizes the prompts used for evaluation and the initial text provided to the LLM, significantly improving accuracy on complex question-answering benchmarks like BBH, MMLU, and GPQA. The research demonstrates the potential of using meta-learning to create more adaptable and efficient LLM-based optimization systems.