Learning to Explore: An In-Context Learning Approach for Pure Exploration

This paper introduces In-Context Pure Exploration (ICPE), a novel deep learning framework designed to learn exploration strategies for active sequential hypothesis testing. Unlike traditional methods that rely on explicit problem-specific algorithms, ICPE uses a Transformer architecture to infer the underlying problem structure directly from experience. The framework combines supervised and reinforcement learning, enabling agents to efficiently discover effective sampling techniques for identifying an unknown property of an environment. The authors validate ICPE through various experiments, including Multi-Armed Bandit (MAB) problems, semi-synthetic pixel sampling for image classification, Magic Room navigation, and exploration on feedback graphs, demonstrating its ability to perform well in diverse, structured environments. The results indicate that ICPE can autonomously learn efficient exploration policies, even rediscovering algorithms like binary search, highlighting its potential as a data-efficient and practical method for exploration in complex settings.