SAM & SAM 2 for Medical Image Segmentation.

Due to the inherent flexibility of prompting, foundation models have emerged as the predominant force in the fields of natural language processing and computer vision. The introduction of the Segment Anything Model (SAM) (paper) and SAM2 (paper) signifies a noteworthy expansion of the prompt-driven paradigm into the domain of image/video segmentation, introducing a plethora of previously unexplored capabilities.
We provide a comprehensive survey of recent endeavors aimed at extending the efficacy of SAM to medical image segmentation tasks, encompassing both empirical benchmarking and methodological adaptations. Additionally, we explore potential avenues for future research directions in SAM's role within medical image segmentation. Please refer to the paper for more details.
This repo will continue to track and summarize the latest research progress of SAM in medical image segmentation to support ongoing research endeavors. If you find this project helpful, please consider stars or citing. Feel free to contact for any suggestions. If you would like to contribute, please open an issue.

@article{SAM4MIS,
  title={Segment Anything Model for Medical Image Segmentation: Current Applications and Future Directions},
  author={Zhang, Yichi and Shen, Zhenrong and Jiao, Rushi},
  journal={Computers in Biology and Medicine},
  volume={171},
  pages={108238},
  year={2024}
}

Last update 2024-8-13

Introduction: About SAM & SAM2
Literature Reviews of SAM 2 for Medical Image Segmentation
Literature Reviews of SAM for Medical Image Segmentation
Large-Scale Datasets for Developing Medical Foundation Models
CVPR2024 Workshop: Segment Anything in Medical Images on Laptop

About Segment Anything Model (SAM) <div id="introduction"></div>

Segment Anything Model (SAM) uses vision transformer-based image encoder to extract image features and compute an image embedding, and prompt encoder to embed prompts and incorporate user interactions. Then extranted information from two encoders are combined to alightweight mask decoder to generate segmentation results based on the image embedding, prompt embedding, and output token. For more details, please refer to the original paper of SAM.

A brief chronology of Segment Anything Model (SAM) and its variants for medical image segmentation in 2023.

Literature Reviews of SAM 2 Adaptions for Medical Image Segmentation. <div id="sam24mis"></div>

Date	Authors	Title	Code
202408	Y. Yamagishi et al.	Zero-shot 3D Segmentation of Abdominal Organs in CT Scans Using Segment Anything Model 2: Adapting Video Tracking Capabilities for 3D Medical Imaging (paper)	None
202408	M. Mansoori et al.	Polyp SAM 2: Advancing Zero shot Polyp Segmentation in Colorectal Cancer Detection (paper)	Code
202408	AS. Yu et al.	Novel adaptation of video segmentation to 3D MRI: efficient zero-shot knee segmentation with SAM2 (paper)	None
202408	J. Yu et al.	SAM 2 in Robotic Surgery: An Empirical Evaluation for Robustness and Generalization in Surgical Video Segmentation (paper)	None
202408	T. Chen et al.	SAM2-Adapter: Evaluating & Adapting Segment Anything 2 in Downstream Tasks: Camouflage, Shadow, Medical Image Segmentation, and More (paper)	None
202408	S. Sengupta et al.	Is SAM 2 Better than SAM in Medical Image Segmentation? (paper)	None
202408	Y. Shen et al.	Performance and Non-adversarial Robustness of the Segment Anything Model 2 in Surgical Video Segmentation (paper)	None
202408	M. Zhang et al.	SAM2-PATH: A better segment anything model for semantic segmentation in digital pathology (paper)	Code
202408	J. Ma et al.	Segment Anything in Medical Images and Videos: Benchmark and Deployment (paper)	Code
202408	Z. Yan et al.	Biomedical SAM 2: Segment Anything in Biomedical Images and Videos (paper)	None
202408	C. Shen et al.	Interactive 3D Medical Image Segmentation with SAM 2 (paper)	Code
202408	A. Lou et al.	Zero-Shot Surgical Tool Segmentation in Monocular Video Using Segment Anything Model 2 (paper)	None
202408	J. Zhu et al.	Medical SAM 2: Segment medical images as video via Segment Anything Model 2 (paper)	Code
202408	H. Dong et al.	Segment anything model 2: an application to 2D and 3D medical images (paper)	None

Literature Reviews of Foundation Models / SAM for Medical Image Segmentation. <div id="sam4mis"></div>

Date	Authors	Title	Code
202408	J. Wei et al.	SAM-FNet: SAM-Guided Fusion Network for Laryngo-Pharyngeal Tumor Detection (paper)	Code
202408	X. Wei et al.	PromptSAM+: Malware Detection based on Prompt Segment Anything Model (paper)	Code
202407	J. Cai et al.	PESAM: Privacy-Enhanced Segment Anything Model for Medical Image Segmentation (paper)	None
202407	M. Asokan et al.	A Federated Learning-Friendly Approach for Parameter-Efficient Fine-Tuning of SAM in 3D Segmentation (paper)	Code
202407	SN. Gowda et al.	CC-SAM: SAM with Cross-feature Attention and Context for Ultrasound Image Segmentation(paper)	None
202407	X. Huo et al.	Dr-SAM: U-Shape Structure Segment Anything Model for Generalizable Medical Image Segmentation (paper)	None
202407	H. Fang et al.	SAM-MIL: A Spatial Contextual Aware Multiple Instance Learning Approach for Whole Slide Image Classification (paper)	None
202407	Q. Xu et al.	ESP-MedSAM: Efficient Self-Prompting SAM for Universal Domain-Generalized Medical Image Segmentation (paper)	Code
202407	X. Zhao et al.	SAM-Driven Weakly Supervised Nodule Segmentation with Uncertainty-Aware Cross Teaching (paper)	None
202407	Q. Xu et al.	ProtoSAM: One Shot Medical Image Segmentation With Foundational Models (paper)	Code
202407	A. Murali et al.	CycleSAM: One-Shot Surgical Scene Segmentation using Cycle-Consistent Feature Matching to Prompt SAM (paper)	None
202407	T. Song et al.	TinySAM-Med3D: A Lightweight Segment Anything Model for Volumetric Medical Imaging with Mixture of Experts (paper)	None
202407	Y. Gao et al.	MBA-Net: SAM-driven Bidirectional Aggregation Network for Ovarian Tumor Segmentation (paper)	None
202407	J. Miao et al.	Cross Prompting Consistency with Segment Anything Model for Semi-supervised Medical Image Segmentation (paper)	Code
202407	G. Wang et al.	SAM-Med3D-MoE: Towards a Non-Forgetting Segment Anything Model via Mixture of Experts for 3D Medical Image Segmentation (paper)	None
202407	Z. Zhang et al.	Quantification of cardiac capillarization in basement-membrane-immunostained myocardial slices using Segment Anything Model (paper)	None
202407	H. Li et al.	ASPS: Augmented Segment Anything Model for Polyp Segmentation (paper)	Code
202406	Y. Xie et al.	SimTxtSeg: Weakly-Supervised Medical Image Segmentation with Simple Text Cues (paper)	None
202406	X. Deng et al.	MemSAM: Taming Segment Anything Model for Echocardiography Video Segmentation (paper)	Code
202406	Yunhe Gao	Training Like a Medical Resident: Context-Prior Learning Toward Universal Medical Image Segmentation (paper)	Code
202406	C.D Albelda et al.	How SAM Perceives Different mp-MRI Brain Tumor Domains? (paper)	Code
202406	T. Huang et al.	Improving Segment Anything on the Fly: Auxiliary Online Learning and Adaptive Fusion for Medical Image Segmentation (paper)	Code
202406	B. Towle et al.	SimSAM: Zero-shot Medical Image Segmentation via Simulated Interaction (paper)	Code
202405	Y. Gu et al.	LeSAM: Adapt Segment Anything Model for medical lesion segmentation (paper)	None
202405	J. Leng et al.	Development of UroSAM: A Machine Learning Model to Automatically Identify Kidney Stone Composition from Endoscopic Video (paper)	None
202405	MM. Rahman et al.	PP-SAM: Perturbed Prompts for Robust Adaptation of Segment Anything Model for Polyp Segmentation (paper)	Code
202405	X. Zhang et al.	A Foundation Model for Brain Lesion Segmentation with Mixture of Modality Experts (paper)	Code
202405	TJ. Chan et al.	SAM3D: Zero-Shot Semi-Automatic Segmentation in 3D Medical Images with the Segment Anything Model (paper)	None
202405	HL. Zedda et al.	SAMMI: Segment Anything Model for Malaria Identification (paper)	None
202404	H. Zhou et al.	AGSAM: Agent-Guided Segment Anything Model for Automatic Segmentation in Few-Shot Scenarios (paper)	None
202404	V. Zohranyan et al.	Dr-SAM: An End-to-End Framework for Vascular Segmentation, Diameter Estimation, and Anomaly Detection on Angiography Images (paper)	Code
202404	Z. Tu et al.	Ultrasound SAM Adapter: Adapting SAM for Breast Lesion Segmentation in Ultrasound Images (paper)	Code
202404	Y. Sheng et al.	Surgical-DeSAM: Decoupling SAM for Instrument Segmentation in Robotic Surgery (paper)	None
202404	J. Yu et al.	Adapting SAM for Surgical Instrument Tracking and Segmentation in Endoscopic Submucosal Dissection Videos (paper)	None
202404	H. Gu et al.	How to build the best medical image segmentation algorithm using foundation models: a comprehensive empirical study with Segment Anything Model (paper)	Code
202404	W. Abebe et al.	SAM-I-Am: Semantic Boosting for Zero-shot Atomic-Scale Electron Micrograph Segmentation (paper)	None
202404	S. Aleem et al.	Test-Time Adaptation with SaLIP: A Cascade of SAM and CLIP for Zero-shot Medical Image Segmentation (paper)	Code
202404	Z. Su et al.	Adapting SAM to histopathology images for tumor bud segmentation in colorectal cancer (paper)	None
202404	Y. Ding et al.	Barely-supervised Brain Tumor Segmentation via Employing Segment Anything Model (paper)	None
202404	Y. Zhu et al.	SAM-Att: A Prompt-free SAM-related Model with an Attention Module for Automatic Segmentation of the Left Ventricle in Echocardiography (paper)	None
202404	Y.