Towards Global Optimal Visual In-Context Learning Prompt Selection

This research introduces a novel framework for Visual In-Context Learning (VICL), a method where artificial intelligence models learn from provided visual examples. The primary focus is on optimizing the selection of these "in-context examples," which significantly impacts the model's performance on tasks like image segmentation, object detection, and colorization. The authors propose a transformer-based list-wise ranker to identify the most relevant examples, overcoming limitations of previous pair-wise ranking methods that often rely on visual similarity. Furthermore, a consistency-aware ranking aggregator is introduced to synthesize more reliable global rankings from the partial predictions of the ranker. Extensive experiments demonstrate that this new approach consistently outperforms existing methods, leading to state-of-the-art results across various visual tasks.