Retrieval-Augmented Representation for Semi-Supervised Medical Image Segmentation.

Abstract

Semi-supervised learning (SSL) offers a promising solution to reduce annotation costs in medical image segmentation. Recent text-enhanced SSL methods incorporate domain-specific textual cues to improve representation learning. However, they often overemphasize language priors and neglect the importance of visual features for precise segmentation. In this work, we propose Retrieval-Augmented Representation (RAR), a novel framework that enriches the representation of unlabeled data by retrieving semantically relevant labeled samples from both visual and textual perspectives. Rather than fusing multimodal features, RAR treats the retrieved samples as positive anchors in a contrastive learning framework, encouraging the model to learn robust and spatially coherent features guided by both semantic context and anatomical structure. Extensive experiments across five diverse modalities (CT, X-ray, MRI, ultrasound, and pathology) show that RAR consistently outperforms state-of-the-art vision-only and text-enhanced SSL approaches, achieving an average Dice improvement of 6.2% over the strongest baseline across the evaluated tasks, while maintaining competitive computational efficiency. Our results suggest the robustness and generalizability of RAR. Code is available at:https://github.com/jgfiuuuu/RAR.