Advances in Medical Image Analysis with Vision Transformers: A Comprehensive Review

:fire::fire:This is a collection of awesome articles about Transformer models in medical imaging :fire::fire:

:loudspeaker: Our review paper published on MedIA: Advances in Medical Image Analysis with Vision Transformers: A Comprehensive Review :heart:

:loudspeaker: Our review paper published on arXiv: Advances in Medical Image Analysis with Vision Transformers: A Comprehensive Review :heart:

Citation

@article{azad2023advances,
  title={Advances in Medical Image Analysis with Vision Transformers: A Comprehensive Review},
  author={Azad, Reza and Kazerouni, Amirhossein and Heidari, Moein and Aghdam, Ehsan Khodapanah and Molaei, Amirali and Jia, Yiwei and Jose, Abin and Roy, Rijo and Merhof, Dorit},
  journal={Medical Image Analysis},
  volume = {91},
  pages={103000},
  year={2024},
  issn = {1361-8415},
  publisher={Elsevier}
}

Taxonomy
Papers

Taxonomy

Transformers

Papers

Image Classification

HATNet: An End-to-End Holistic Attention Network for Diagnosis of Breast Biopsy Images Sachin Mehta, Ximing Lu, Donald Weaver, Joann G. Elmore, Hannaneh Hajishirzi, Linda Shapiro [25th Jul, 2020] [MedIA Journal, 2022]
[PDF] [GitHub]

A graph-transformer for whole slide image classification Yi Zheng, Rushin H. Gindra, Emily J. Green, Eric J. Burks, Margrit Betke, Jennifer E. Beane, Vijaya B. Kolachalama [19th May, 2022] [TMI Journal, 2022]
[PDF] [GitHub]

RadioTransformer: A Cascaded Global-Focal Transformer for Visual Attention-guided Disease Classification Moinak Bhattacharya, Shubham Jain, Prateek Prasanna [23rd Feb., 2022] [ECCV, 2022]
[PDF]

Federated Split Vision Transformer for COVID-19 CXR Diagnosis using Task-Agnostic Training Sangjoon Park, Gwanghyun Kim, Jeongsol Kim, Boah Kim, Jong Chul Ye [2nd Nov., 2021] [NeurIPS, 2021]
[PDF]

Vision transformer for classification of breast ultrasound images Behnaz Gheflati, Hassan Rivaz [27th Oct., 2021] [EMBC, 2022]
[PDF]

MIL-VT: Multiple Instance Learning Enhanced Vision Transformer for Fundus Image Classification Shuang Yu, Kai Ma, Qi Bi, Cheng Bian, Munan Ning, Nanjun He, Yuexiang Li, Hanruo Liu, Yefeng Zheng [21st Sep., 2021] [MICCAI, 2021]
[PDF] [GitHub]

3DMeT: 3D Medical Image Transformer for Knee Cartilage Defect Assessment Sheng Wang, Zixu Zhuang, Kai Xuan, Dahong Qian, Zhong Xue, Jia Xu, Ying Liu, Yiming Chai, Lichi Zhang, Qian Wang, Dinggang Shen [21st Sep., 2021] [MICCAI Workshop, 2021]
[PDF]

COVID-Transformer: Interpretable COVID-19 Detection Using Vision Transformer for Healthcare Debaditya Shome, T. Kar, Sachi Nandan Mohanty, Prayag Tiwari, Khan Muhammad, Abdullah AlTameem, Yazhou Zhang, Abdul Khader Jilani Saudagar [23rd Sep., 2021] [International Journal of Environmental Research and Public Health, 2021]
[PDF] [GitHub]

Is it Time to Replace CNNs with Transformers for Medical Images? Christos Matsoukas, Johan Fredin Haslum, Magnus Söderberg, Kevin Smith [20th Aug., 2021] [ICCV Workshop, 2021]
[PDF] [GitHub]

Vision Transformer for femur fracture classification Leonardo Tanzi, Andrea Audisio, Giansalvo Cirrincione, Alessandro Aprato, Enrico Vezzetti [7th Aug., 2021] [Injury Journal, 2022]
[PDF]

xViTCOS: Explainable Vision Transformer Based COVID-19 Screening Using Radiography Arnab Kumar Mondal, Arnab Bhattacharjee, Parag Singla, A. P. Prathosh [7th Jul., 2021] [IEEE Journal of Translational Engineering in Health and Medicine, 2021]
[PDF] [Github]

COVID-VIT: Classification of COVID-19 from CT chest images based on vision transformer models Xiaohong Gao, Yu Qian, Alice Gao [4th Jul., 2021] [NextComp, 2022]
[PDF] [GitHub]

TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification Zhuchen Shao, Hao Bian, Yang Chen, Yifeng Wang, Jian Zhang, Xiangyang Ji, Yongbing Zhang [2nd Jun., 2021] [NeurIPS, 2021]
[PDF] [GitHub]

Lesion-Aware Transformers for Diabetic Retinopathy Grading Rui Sun, Yihao Li, Tianzhu Zhang, Zhendong Mao, Feng Wu, Yongdong Zhang [1st Jun., 2021] [CVPR, 2021]
[PDF]

POCFormer: A Lightweight Transformer Architecture for Detection of COVID-19 Using Point of Care Ultrasound Shehan Perera, Srikar Adhikari, Alper Yilmaz [20th May, 2021] [ICIP, 2022]
[PDF]

Automatic diagnosis of covid-19 using a tailored transformer-like network Chengeng Liu, Qingshan Yin [21st Apr., 2021] [CISAT, 2021]
[PDF]

Vision Transformer for COVID-19 CXR Diagnosis using Chest X-ray Feature Corpus Sangjoon Park, Gwanghyun Kim, Yujin Oh, Joon Beom Seo, Sang Min Lee, Jin Hwan Kim, Sungjun Moon, Jae-Kwang Lim, Jong Chul Ye [12th Mar., 2021] [arXiv, 2021]
[PDF]

TransMed: Transformers Advance Multi-modal Medical Image Classification Yin Dai, Yifan Gao [10th Mar., 2021] [Diagnostics, 2021]
[PDF]

Image Segmentation

TransDeepLab: Convolution-Free Transformer-based DeepLab v3+ for Medical Image Segmentation Reza Azad, Moein Heidari, Moein Shariatnia, Ehsan Khodapanah Aghdam, Sanaz Karimijafarbigloo, Ehsan Adeli, Dorit Merhof [1st Aug., 2022] [MICCAI Workshop, 2022]
[PDF] [GitHub]

HiFormer: Hierarchical Multi-scale Representations Using Transformers for Medical Image Segmentation Moein Heidari, Amirhossein Kazerouni, Milad Soltany, Reza Azad, Ehsan Khodapanah, Aghdam Julien Cohen-Adad, Dorit Merhof [18th Jul., 2022] [WACV, 2023]
[PDF] [GitHub]

Self Pre-training with Masked Autoencoders for Medical Image Analysis Lei Zhou, Huidong Liu, Joseph Bae, Junjun He, Dimitris Samaras, Prateek Prasanna [10th Mar., 2022] [arXiv, 2022]
[PDF]

Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images Ali Hatamizadeh, Vishwesh Nath, Yucheng Tang, Dong Yang, Holger Roth, Daguang Xu [4th Jan., 2022] [MICCAI Workshop]
[PDF] [GitHub]

Semi-Supervised Medical Image Segmentation via Cross Teaching between CNN and Transformer Xiangde Luo, Minhao Hu, Tao Song, Guotai Wang, Shaoting Zhang [9th Dec., 2021] [MIDL, 2022]
[PDF] [Github]

T-AutoML: Automated Machine Learning for Lesion Segmentation using Transformers in 3D Medical Imaging Dong Yang, Andriy Myronenko, Xiaosong Wang, Ziyue Xu, Holger R. Roth, Daguang Xu [15th Nov., 2021] [ICCV, 2021]
[PDF]

MISSFormer: An Effective Medical Image Segmentation Transformer Xiaohong Huang, Zhifang Deng, Dandan Li, Xueguang Yuan [15th Sep., 2021] [TMI Journal, 2022]
[PDF] [GitHub]

nnFormer: Interleaved Transformer for Volumetric Segmentation Hong-Yu Zhou, Jiansen Guo, Yinghao Zhang, Lequan Yu, Liansheng Wang, Yizhou Yu [7th Sep., 2021] [arXiv, 2021]
[PDF] [GitHub]

Medical Image Segmentation Using Squeeze-and-Expansion Transformers Shaohua Li, Xiuchao Sui, Xiangde Luo, Xinxing Xu, Yong Liu, Rick Goh [20th May, 2021] [IJCAI, 2021]
[PDF] [GitHub]

Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation Hu Cao, Yueyue Wang, Joy Chen, Dongsheng Jiang, Xiaopeng Zhang, Qi Tian, Manning Wang [12th May, 2021] [arXiv, 2021]
[PDF] [GitHub]

UNETR: Transformers for 3D Medical Image Segmentation Ali Hatamizadeh, Yucheng Tang, Vishwesh Nath, Dong Yang, Andriy Myronenko, Bennett Landman, Holger Roth, Daguang Xu [18th Mar., 2021] [WACV, 2022]
[PDF] [GitHub]

TransBTS: Multimodal Brain Tumor Segmentation Using Transformer Jiangyun Li, Wenxuan Wang, Chen Chen, Tianxiang Zhang, Sen Zha, Hong Yu, Jing Wang [7th Mar, 2021] [MICCAI, 2021]
[PDF] [GitHub]

CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation Yutong Xie, Jianpeng Zhang, Chunhua Shen, Yong Xia [4th Mar., 2021] [MICCAI, 2021]
[PDF] [GitHub]

Medical Transformer: Gated Axial-Attention for Medical Image Segmentation Jeya Maria Jose Valanarasu, Poojan Oza, Ilker Hacihaliloglu, Vishal M. Patel [21th Feb., 2021] [MICCAI, 2021]
[PDF] [GitHub]

TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation *Yundong Zhang, Huiye Liu, Qiang