Towards Global Optimal Visual In-Context Learning Prompt Selection

This research introduces Partial2Global, a novel framework for Visual In-Context Learning (VICL), focusing on the critical task of selecting optimal "in-context examples" to enhance the performance of visual foundation models. The paper highlights that randomly chosen examples often yield poor results, and that visual similarity alone is an unreliable metric for selection. Partial2Global addresses these challenges by employing a transformer-based list-wise ranker for a more comprehensive comparison of example alternatives, coupled with a consistency-aware ranking aggregator that ensures globally consistent rankings. Through experiments on tasks like foreground segmentation and object detection, the authors demonstrate that their method consistently outperforms existing techniques, achieving new state-of-the-art results by providing superior in-context examples. The framework effectively mitigates limitations of previous approaches, which struggled with incomplete information or inconsistent ranking predictions.