Exploration and Exploitation Errors Are Measurable for Language Model Agents

This research paper introduces a systematic framework to measure how Language Model (LM) agents balance exploration and exploitation in complex, open-ended environments. The authors designed a policy-agnostic metric that identifies structural errors in an agent's trajectory without needing a reference solution, distinguishing between redundant movement and failed knowledge application. Their experiments utilize partially observable grid maps paired with symbolic task graphs to ensure models reason purely from environmental data rather than relying on prior training knowledge. Findings reveal that while reasoning-heavy models perform better, even top-tier agents struggle with these tasks, though performance can be boosted through harness engineering. Ultimately, the study demonstrates a strong correlation between low exploration errors and overall task success, providing a new benchmark for agentic AI development.