Anomaly Detection Learning Resources

.. image:: https://img.shields.io/github/stars/yzhao062/anomaly-detection-resources.svg :target: https://github.com/yzhao062/anomaly-detection-resources/stargazers :alt: GitHub stars

.. image:: https://img.shields.io/github/forks/yzhao062/anomaly-detection-resources.svg?color=blue :target: https://github.com/yzhao062/anomaly-detection-resources/network :alt: GitHub forks

.. image:: https://img.shields.io/github/license/yzhao062/anomaly-detection-resources.svg?color=blue :target: https://github.com/yzhao062/anomaly-detection-resources/blob/master/LICENSE :alt: License

.. image:: https://awesome.re/badge-flat2.svg :target: https://awesome.re/badge-flat2.svg :alt: Awesome

.. image:: https://img.shields.io/badge/ADBench-benchmark_results-pink :target: https://github.com/Minqi824/ADBench :alt: Benchmark

Outlier Detection <https://en.wikipedia.org/wiki/Anomaly_detection>_ (also known as Anomaly Detection) is an exciting yet challenging field, which aims to identify outlying objects that are deviant from the general data distribution. Outlier detection has been proven critical in many fields, such as credit card fraud analytics, network intrusion detection, and mechanical unit defect detection.

This repository collects:

#. Books & Academic Papers #. Online Courses and Videos #. Outlier Datasets #. Open-source and Commercial Libraries/Toolkits #. Key Conferences & Journals

More items will be added to the repository. Please feel free to suggest other key resources by opening an issue report, submitting a pull request, or dropping me an email @ (yzhao010@usc.edu). Enjoy reading!

BTW, you may find my [GitHub] <https://github.com/yzhao062>_ and [outlier detection papers] <https://scholar.google.com/citations?user=zoGDYsoAAAAJ&hl=en>_ useful, especially PyOD library <https://github.com/yzhao062/pyod>_ and ADBench benchmark <https://github.com/Minqi824/ADBench>_.

1. Books & Tutorials & Benchmarks <#1-books--tutorials--benchmarks>_
- 1.1. Books <#11-books>_
- 1.2. Tutorials <#12-tutorials>_
- 1.3. Benchmarks <#13-benchmarks>_
2. Courses/Seminars/Videos <#2-coursesseminarsvideos>_
3. Toolbox & Datasets <#3-toolbox--datasets>_
- 3.1. Multivariate data outlier detection <#31-multivariate-data>_
- 3.2. Time series outlier detection <#32-time-series-outlier-detection>_
- 3.3. Graph Outlier Detection <#33-graph-outlier-detection>_
- 3.4. Real-time Elasticsearch <#34-real-time-elasticsearch>_
- 3.5. Datasets <#35-datasets>_
4. Papers <#4-papers>_
- 4.1. Overview & Survey Papers <#41-overview--survey-papers>_
- 4.2. Key Algorithms <#42-key-algorithms>_
- 4.3. Graph & Network Outlier Detection <#43-graph--network-outlier-detection>_
- 4.4. Time Series Outlier Detection <#44-time-series-outlier-detection>_
- 4.5. Feature Selection in Outlier Detection <#45-feature-selection-in-outlier-detection>_
- 4.6. High-dimensional & Subspace Outliers <#46-high-dimensional--subspace-outliers>_
- 4.7. Outlier Ensembles <#47-outlier-ensembles>_
- 4.8. Outlier Detection in Evolving Data <#48-outlier-detection-in-evolving-data>_
- 4.9. Representation Learning in Outlier Detection <#49-representation-learning-in-outlier-detection>_
- 4.10. Interpretability <#410-interpretability>_
- 4.11. Outlier Detection with Neural Networks <#411-outlier-detection-with-neural-networks>_
- 4.12. Active Anomaly Detection <#412-active-anomaly-detection>_
- 4.13. Interactive Outlier Detection <#413-interactive-outlier-detection>_
- 4.14. Outlier Detection in Other fields <#414-outlier-detection-in-other-fields>_
- 4.15. Outlier Detection Applications <#415-outlier-detection-applications>_
- 4.16. Automated Outlier Detection <#416-automated-outlier-detection>_
- 4.17. Machine Learning Systems for Outlier Detection <#417-machine-learning-systems-for-outlier-detection>_
- 4.18. Fairness and Bias in Outlier Detection <#418-fairness-and-bias-in-outlier-detection>_
- 4.19. Isolation-based Methods <#419-isolation-based-methods>_
- 4.20. Emerging and Interesting Topics <#420-emerging-and-interesting-topics>_
5. Key Conferences/Workshops/Journals <#5-key-conferencesworkshopsjournals>_
- 5.1. Conferences & Workshops <#51-conferences--workshops>_
- 5.2. Journals <#52-journals>_

Books & Tutorials & Benchmarks

1.1. Books ^^^^^^^^^^

Outlier Analysis <https://link.springer.com/book/10.1007/978-3-319-47578-3>_ by Charu Aggarwal: Classical text book covering most of the outlier analysis techniques. A must-read for people in the field of outlier detection. [Preview.pdf] <http://charuaggarwal.net/outlierbook.pdf>_

Outlier Ensembles: An Introduction <https://www.springer.com/gp/book/9783319547640>_ by Charu Aggarwal and Saket Sathe: Great intro book for ensemble learning in outlier analysis.

Data Mining: Concepts and Techniques (3rd) <https://www.elsevier.com/books/data-mining-concepts-and-techniques/han/978-0-12-381479-1>_ by Jiawei Han and Micheline Kamber and Jian Pei: Chapter 12 discusses outlier detection with many key points. [Google Search] <https://www.google.ca/search?&q=data+mining+jiawei+han&oq=data+ming+jiawei>_

1.2. Tutorials ^^^^^^^^^^^^^^

===================================================== ============================================ ===== ============================ ========================================================================================================================================================================== Tutorial Title Venue Year Ref Materials ===================================================== ============================================ ===== ============================ ========================================================================================================================================================================== Data mining for anomaly detection PKDD 2008 [#Lazarevic2008Data]_ [Video] <http://videolectures.net/ecmlpkdd08_lazarevic_dmfa/>_ Outlier detection techniques ACM SIGKDD 2010 [#Kriegel2010Outlier]_ [PDF] <https://imada.sdu.dk/~zimek/publications/KDD2010/kdd10-outlier-tutorial.pdf>_ Anomaly Detection: A Tutorial ICDM 2011 [#Chawla2011Anomaly]_ [PDF] <http://webdocs.cs.ualberta.ca/~icdm2011/downloads/ICDM2011_anomaly_detection_tutorial.pdf>_ Anomaly Detection in Networks KDD 2017 [#Mendiratta2017Anomaly]_ [Page] <https://veena-mendiratta.blog/tutorial-anomaly-detection-in-networks/>_ Which Outlier Detector Should I use? ICDM 2018 [#Ting2018Which]_ [PDF] <https://ieeexplore.ieee.org/document/8594824>_ Deep Learning for Anomaly Detection KDD 2020 [#Wang2020Deep]_ [HTML] <https://sites.google.com/view/kdd2020deepeye/home>, [Video] <https://www.youtube.com/watch?v=Fn0qDbKL3UI&list=PLn0nrSd4xjja7AD3aY9Jxmr820gx59EQC&index=66> Deep Learning for Anomaly Detection WSDM 2021 [#Pang2021Deep]_ [HTML] <https://sites.google.com/site/gspangsite/wsdm21_tutorial>_ Toward Explainable Deep Anomaly Detection KDD 2021 [#Pang2021Toward]_ [HTML] <https://sites.google.com/site/gspangsite/kdd21_tutorial>_ Recent Advances in Anomaly Detection CVPR 2023 [#Pang2023recent]_ [HTML] <https://sites.google.com/view/cvpr2023-tutorial-on-ad/>, [Video] <https://www.youtube.com/watch?v=dXxrzWeybBo&feature=youtu.be> Trustworthy Anomaly Detection SDM 2024 [#Yuan2024Trustworthy]_ [HTML] <https://yuan.shuhan.org/talks/SDM24/>_ ===================================================== ============================================ ===== ============================ ==========================================================================================================================================================================

1.3. Benchmarks ^^^^^^^^^^^^^^^

News: We just released a 36-page, the most comprehensive anomaly detection benchmark paper <https://www.andrew.cmu.edu/user/yuezhao2/papers/22-preprint-adbench.pdf>. The fully open-sourced ADBench <https://github.com/Minqi824/ADBench> compares 30 anomaly detection algorithms on 55 benchmark datasets.

============= ================================================================================================= ============================ ===== ============================ ========================================================================================================================================================================== Data Types Paper Title Venue Year Ref Materials ============= ================================================================================================= ============================ ===== ============================ ========================================================================================================================================================================== Time-series Revisiting Time Series Outlier Detection: Definitions and Benchmarks NeurIPS 2021 [#Lai2021Revisiting]_ [PDF] <https://openreview.net/pdf?id=r8IvOsnHchr>, [Code] <https://github.com/datamllab/tods/tree/benchmark> Graph Benchmarking Node Outlier Detection on Graphs NeurIPS 2022 [#Liu2022Benchmarking]_ [PDF] <https://arxiv.org/abs/2206.10071>, [Code] <https://github.com/pygod-team/pygod/tree/main/benchmark> Graph GADBench: Revisiting and Benchmarking Supervised Graph Anomaly Detection NeurIPS 2023 [#Tang2023GADBench]_ [PDF] <https://arxiv.org/abs/2306.12251>, [Code] <https://github.com/squareRoot3/GADBench> Tabular ADBench: Anomaly Detection Benchmark NeurIPS 2022 [#Han2022Adbench]_ [PDF] <https://arxiv.org/abs/2206.09426>, [Code] <https://github.com/Minqi824/ADBench> Tabular ADGym: Design Choices for Deep Anomaly Detection NeurIPS 2023 [#Jiang2023adgym]_ [PDF] <https://arxiv.org/abs/2309.15376>, [Code] <https://github.com/Minqi824/ADGym> ============= ================================================================================================= ============================ ===== ============================ ==========================================================================================================================================================================

Courses/Seminars/Videos

Coursera Introduction to Anomaly Detection (by IBM)\ : [See Video] <https://www.coursera.org/learn/ai/lecture/ASPv0/introduction-to-anomaly-detection>_

Get started with the Anomaly Detection API (by IBM)\ : [See Website] <https://developer.ibm.com/learningpaths/get-started-anomaly-detection-api/>_

Practical Anomaly Detection by appliedAI Institute: [See Website] <https://transferlab.ai/trainings/practical-anomaly-detection/>, [See Video] <https://www.youtube.com/watch?v=sEoMIDARpJ0&list=PLz6xKPm1Bnd6cDDgct3MDhNWJuPXzsmyW>, [See GitHub] <https://github.com/aai-institute/tfl-training-practical-anomaly-detection>_

Coursera Real-Time Cyber Threat Detection and Mitigation partly covers the topic\ : [See Video] <https://www.coursera.org/learn/real-time-cyber-threat-detection>_

Coursera Machine Learning by Andrew Ng also partly covers the topic\ :

Anomaly Detection vs. Supervised Learning <https://www.coursera.org/learn/machine-learning/lecture/Rkc5x/anomaly-detection-vs-supervised-learning>_
Developing and Evaluating an Anomaly Detection System <https://www.coursera.org/learn/machine-learning/lecture/Mwrni/developing-and-evaluating-an-anomaly-detection-system>_

Udemy Outlier Detection Algorithms in Data Mining and Data Science\ : [See Video] <https://www.udemy.com/outlier-detection-techniques/>_

Stanford Data Mining for Cyber Security also covers part of anomaly detection techniques\ : [See Video] <http://web.stanford.edu/class/cs259d/>_

Toolbox & Datasets

3.1. Multivariate Data ^^^^^^^^^^^^^^^^^^^^^^

[Python] Python Outlier Detection (PyOD) <https://github.com/yzhao062/pyod>_\ : PyOD is a comprehensive and scalable Python toolkit for detecting outlying objects in multivariate data. It contains more than 20 detection algorithms, including emerging deep learning models and outlier ensembles.

[Python, GPU] TOD: Tensor-based Outlier Detection (PyTOD) <https://github.com/yzhao062/pytod>_: A general GPU-accelerated framework for outlier detection.

[Python] Python Streaming Anomaly Detection (PySAD) <https://github.com/selimfirat/pysad>_\ : PySAD is a streaming anomaly detection framework in Python, which provides a complete set of tools for anomaly detection experiments. It currently contains more than 15 online anomaly detection algorithms and 2 different methods to integrate PyOD detectors to the streaming setting.

[Python] Scikit-learn Novelty and Outlier Detection <http://scikit-learn.org/stable/modules/outlier_detection.html>_. It supports some popular algorithms like LOF, Isolation Forest, and One-class SVM.

[Python] Scalable Unsupervised Outlier Detection (SUOD) <https://github.com/yzhao062/suod>_\ : SUOD (Scalable Unsupervised Outlier Detection) is an acceleration framework for large-scale unsupervised outlier detector training and prediction, on top of PyOD.

[Julia] OutlierDetection.jl <https://github.com/OutlierDetectionJL/OutlierDetection.jl>_\ : OutlierDetection.jl is a Julia toolkit for detecting outlying objects, also known as anomalies.

[Java] ELKI: Environment for Developing KDD-Applications Supported by Index-Structures <https://elki-project.github.io/>_\ : ELKI is an open source (AGPLv3) data mining software written in Java. The focus of ELKI is research in algorithms, with an emphasis on unsupervised methods in cluster analysis and outlier detection.

[Java] RapidMiner Anomaly Detection Extension <https://github.com/Markus-Go/rapidminer-anomalydetection>_\ : The Anomaly Detection Extension for RapidMiner comprises the most well know unsupervised anomaly detection algorithms, assigning individual anomaly scores to data rows of example sets. It allows you to find data, which is significantly different from the normal, without the need for the data being labeled.

[R] CRAN Task View: Anomaly Detection with R <https://github.com/pridiltal/ctv-AnomalyDetection>_\ : This CRAN task view contains a list of packages that can be used for anomaly detection with R.

[R] outliers package <https://cran.r-project.org/web/packages/outliers/index.html>_\ : A collection of some tests commonly used for identifying outliers in R.

[Matlab] Anomaly Detection Toolbox - Beta <http://dsmi-lab-ntust.github.io/AnomalyDetectionToolbox/>_\ : A collection of popular outlier detection algorithms in Matlab.

3.2. Time Series Outlier Detection ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[Python] TODS <https://github.com/datamllab/tods>_\ : TODS is a full-stack automated machine learning system for outlier detection on multivariate time-series data.

[Python] skyline <https://github.com/earthgecko/skyline>_\ : Skyline is a near real time anomaly detection system.

[Python] banpei <https://github.com/tsurubee/banpei>_\ : Banpei is a Python package of the anomaly detection.

[Python] telemanom <https://github.com/khundman/telemanom>_\ : A framework for using LSTMs to detect anomalies in multivariate time series data.

[Python] `DeepADoTS