Deep Learning for Polyp Detection and Classification in Colonoscopy

This repository was created from the following review paper: A. Nogueira-Rodríguez; R. Domínguez-Carbajales; H. López-Fernández; Á. Iglesias; J. Cubiella; F. Fdez-Riverola; M. Reboiro-Jato; D. Glez-Peña (2020) Deep Neural Networks approaches for detecting and classifying colorectal polyps. Neurocomputing.

Please, cite it if you find it useful for your research.

AI4PolypNet

As part of AI4PolypNet, we are involved in a challenge that will be developed at iSMIT (September 2024). In this edition we will focus only on colonoscopy images and, apart from classical polyp detection and segmentation we present an extended version of polyp classification, including the challenging serrated sessile adenoma class. All the information is available here.

About this repository

This repository collects the most relevant studies applying Deep Learning for Polyp Detection and Classification in Colonoscopy from a technical point of view, focusing on the low-level details for the implementation of the DL models. In first place, each study is categorized in three types: (i) polyp detection and localization (through bounding boxes or binary masks, i.e. segmentation), (ii) polyp classification, and (iii) simultaneous polyp detection and classification (i.e. studies based on the usage of a single model such as YOLO or SSD to performs simultaneous polyp detection and classification). Secondly, a summary of the public datasets available as well as the private datasets used in the studies is provided. The third section focuses on technical aspects such as the Deep Learning architectures, the data augmentation techniques and the libraries and frameworks used. Finally, the fourth section summarizes the performance metrics reported by each study.

Suggestions are welcome, please check the contribution guidelines before submitting a pull request.

Table of Contents:

Research

Polyp Detection and Localization

Study	Date	Endoscopy type	Imaging technology	Localization type	Multiple polyp	Real time
Tajbakhsh et al. 2014, Tajbakhsh et al. 2015	Sept. 2014 / Apr. 2015	Conventional	N/A	Bounding box	No	Yes
Zhu R. et al. 2015	Oct. 2015	Conventional	N/A	Bounding box (16x16 patches)	Yes	No
Park and Sargent 2016	March 2016	Conventional	NBI, WL	Bounding box	No	No
Yu et al. 2017	Jan. 2017	Conventional	NBI, WL	Bounding box	No	No
Zhang R. et al. 2017	Jan. 2017	Conventional	NBI, WL	No	No	No
Yuan and Meng 2017	Feb. 2017	WCE	N/A	No	No	No
Brandao et al. 2018	Feb. 2018	Conventional/WCE	N/A	Binary mask	Yes	No
Zhang R. et al. 2018	May 2018	Conventional	WL	Bounding box	No	No
Misawa et al. 2018	June 2018	Conventional	WL	No	Yes	No
Zheng Y. et al. 2018	July 2018	Conventional	NBI, WL	Bounding box	Yes	Yes
Shin Y. et al. 2018	July 2018	Conventional	WL	Bounding box	Yes	No
Urban et al. 2018	Sep. 2018	Conventional	NBI, WL	Bounding box	No	Yes
Mohammed et al. 2018, GitHub	Sep. 2018	Conventional	WL	Binary mask	Yes	Yes
Wang et al. 2018, Wang et al. 2018	Oct. 2018	Conventional	N/A	Binary mask	Yes	Yes
Qadir et al. 2019	Apr. 2019	Conventional	NBI, WL	Bounding box	Yes	No
Blanes-Vidal et al. 2019	March 2019	WCE	N/A	Bounding box	Yes	No
Zhang X. et al. 2019	March 2019	Conventional	N/A	Bounding box	Yes	Yes
Misawa et al. 2019	June 2019	Conventional	N/A	No	Yes	No
Zhu X. et al. 2019	June 2019	Conventional	N/A	No	No	Yes
Ahmad et al. 2019	June 2019	Conventional	WL	Bounding box	Yes	Yes
Sornapudi et al. 2019	June 2019	Conventional/WCE	N/A	Binary mask	Yes	No
Wittenberg et al. 2019	Sept. 2019	Conventional	WL	Binary mask	Yes	No
Yuan Y. et al. 2019	Sept. 2019	WCE	N/A	No	No	No
Ma Y. et al. 2019	Oct. 2019	Conventional	N/A	Bounding box	Yes	No
Tashk et al. 2019	Dec. 2019	Conventional	N/A	Binary mask	No	No
Jia X. et al. 2020	Jan. 2020	Conventional	N/A	Binary mask	Yes	No
Ma Y. et al. 2020	May 2020	Conventional	N/A	Bounding box	Yes	No
Young Lee J. et al. 2020	May 2020	Conventional	N/A	Bounding box	Yes	Yes
Wang W. et al. 2020	July 2020	Conventional	WL	No	No	No
Li T. et al. 2020	Oct. 2020	Conventional	N/A	No	No	No
Sánchez-Peralta et al. 2020	Nov. 2020	Conventional	NBI, WL	Binary mask	No	No
Podlasek J. et al. 2020	Dec. 2020	Conventional	N/A	Bounding box	No	Yes
Qadir et al. 2021	Feb. 2021	Conventional	WL	Bounding box	Yes	Yes
Xu J. et al. 2021	Feb. 2021	Conventional	WL	Bounding box	Yes	Yes
Misawa et al. 2021	Apr. 2021	Conventional	WL	No	Yes	Yes
Livovsky et al. 2021	June 2021	Conventional	N/A	Bounding box	Yes	Yes
Pacal et al. 2021	July 2021	Conventional	WL	Bounding box	Yes	Yes
Liu et al. 2021	July 2021	Conventional	N/A	Bounding box	Yes	Yes
Nogueira-Rodríguez et al. 2021	Aug. 2021	Conventional	NBI, WL	Bounding box	Yes	Yes
Yoshida et al. 2021	Aug. 2021	Conventional	WL, LCI	Bounding box	Yes	Yes
Ma Y. et al. 2021	Sep. 2021	Conventional	WL	Bounding box	Yes	No
Pacal et al. 2022	Nov. 2021	Conventional	WL	Bounding box	Yes	Yes
Nogueira-Rodríguez et al. 2022	April 2022	Conventional	NBI, WL	Bounding box	Yes	Yes
Nogueira-Rodríguez et al. 2023	March 2023	Conventional	NBI, WL	Bounding box	Yes	Yes

Polyp Classification

Study	Date	Endoscopy type	Imaging technology	Classes	Real time
Ribeiro et al. 2016	Oct. 2016	Conventional	WL	Neoplastic vs. Non-neoplastic	No
Zhang R. et al. 2017	Jan. 2017	Conventional	NBI, WL	Adenoma vs. hyperplastic <br/> Resectable vs. non-resectable<br/> Adenoma vs. hyperplastic vs. serrated	No
Byrne et al. 2017	Oct. 2017	Conventional	NBI	Adenoma vs. hyperplastic	Yes
Komeda et al. 2017	Dec. 2017	Conventional	NBI, WL, Chromoendoscopy	Adenoma vs. non-adenoma	No
Chen et al. 2018	Feb. 2018	Conventional	NBI	Neoplastic vs. hyperplastic	No
Lui et al. 2019	Apr. 2019	Conventional	NBI, WL	Endoscopically curable lesions vs. endoscopically incurable lesion	No
Kandel et al. 2019	June 2019	Conventional	N/A	Adenoma vs. hyperplastic vs. serrated (sessile serrated adenoma/traditional serrated adenoma)	No
Zachariah et al. 2019	Oct. 2019	Conventional	NBI, WL	Adenoma vs. serrated	Yes
Bour et al. 2019	Dec. 2019	Conventional	N/A	Paris classification: not dangeours (types Ip, Is, IIa, and IIb) vs. dangerous (type IIc) vs. cancer (type III)	No
Patino-Barrientos et al. 2020	Jan. 2020	Conventional	WL	Kudo's classification: malignant (types I, II, III, and IV) vs. non-malignant (type V)	No
Cheng Tao Pu et al. 2020	Feb. 2020	Conventional	NBI, BLI	Modified Sano's (MS) classification: MS I (Hyperplastic) vs. MS II (Low-grade tubular adenomas) vs. MS