Rethinking and Improving Visual Prompt Selection for In-Context Learning Segmentation

This paper introduces a novel Stepwise Context Search (SCS) method designed to enhance In-Context Learning (ICL) based image segmentation. Traditional ICL methods often require extensive annotations or rely on simple similarity sorting for visual prompt selection, which the authors demonstrate can lead to inconsistent performance. The SCS method addresses these limitations by constructing a smaller, more diverse candidate pool of examples through a clustering and sampling strategy, significantly reducing annotation costs. Furthermore, it incorporates an adaptive search module that dynamically selects the most appropriate visual prompts for specific test images, thereby improving the accuracy and stability of segmentation results across various real-world scenarios.