Hate Speech Dataset Catalogue

<!-- **We are working on a [ckan instance](http://ckan.hatespeechdata.com) -- please give it a look! Contributions welcome :)** -->

This page catalogues datasets annotated for hate speech, online abuse, and offensive language. They may be useful for e.g. training a natural language processing system to detect this language.

The list is maintained by Leon Derczynski, Bertie Vidgen, Hannah Rose Kirk, Pica Johansson, Yi-Ling Chung, Mads Guldborg Kjeldgaard Kongsbak, Laila Sprejer, and Philine Zeinert.

We provide a list of datasets and keywords. If you would like to contribute to our catalogue or add your dataset, please see the instructions for contributing.

If you use these resources, please cite (and read!) our paper: Directions in Abusive Language Training Data: Garbage In, Garbage Out. And if you would like to find other resources for researching online hate, visit The Alan Turing Institute's Online Hate Research Hub or read The Alan Turing Institute's Reading List on Online Hate and Abuse Research.

If you're looking for a good paper on online hate training datasets (beyond our paper, of course!) then have a look at 'Resources and benchmark corpora for hate speech detection: a systematic review' by Poletto et al. in Language Resources and Evaluation.

Please send contributions via github pull request. You can do this by visiting the source code on github and clicking the edit icon (a pencil, above the text, on the right) - more details below. There's a commented-out markdown template at the top of this file. Accompanying data statements preferred for all corpora.

<a id="Datasets-header"></a>

Datasets Table of Contents

• Albanian
• Arabic
• Bengali
• Chinese
• Croatian
• Danish
• Dutch
• English
• Estonian
• French
• German
• Greek
• Hindi
• Indonesian
• Italian
• Korean
• Latvian
• Portuguese
• Polish
• Russian
• Slovene
• Spanish
• Turkish
• Ukranian
• Urdu

List of datasets

<!-- dataset template ### TITLE * Link to publication: [url](url) - link to the documentation and/or a data statement about the data * Link to data: [url](url) - direct download is preferred, e.g. a link straight to a .zip file * Task description: How the task is framed in this data, e.g. "Binary (Hate, Not)", "Hierarchical", "Three-class (Hate speech, Offensive language, None)" * Details of task: Free-text description of the task this data models, e.g. "Misogyny detection on social media in Danish" * Size of dataset: Give the number of instances of abusive/non-abusive/other items * Percentage abusive: e.g. 1.2% * Language: e.g. Arabic * Level of annotation: What is an "instance", in this dataset? e.g. Posts, User, Conversation, ... * Platform: e.g. twitter, snapchat, .. * Medium: text / image / audio / ... * Reference: Give a bibliographic reference for the data (if there is one), with title, author, year, venue etc -->

<a id="Albanian-header"></a>

Albanian

Detecting Abusive Albanian

• Link to publication: https://arxiv.org/abs/2107.13592
• Link to data: https://doi.org/10.6084/m9.figshare.19333298.v1
• Task description: Hierarchical (offensive/not; untargeted/targeted; person/group/other)
• Details of task: Detect and categorise abusive language in social media data
• Size of dataset: 11 874
• Percentage abusive: 13.2%
• Language: Albanian
• Level of annotation: Posts
• Platform: Instagram, Youtube
• Medium: Text
• Reference: Nurce, E., Keci, J., Derczynski, L., 2021. Detecting Abusive Albanian. arXiv:2107.13592
• Dataset reader: 🤗 strombergnlp/shaj

<a id="Arabic-header"></a>

Arabic

Let-Mi: An Arabic Levantine Twitter Dataset for Misogynistic Language

• Link to publication: https://arxiv.org/abs/2103.10195
• Link to data: https://drive.google.com/file/d/1mM2vnjsy7QfUmdVUpKqHRJjZyQobhTrW/view
• Task description: Binary (misogyny/none) and Multi-class (none, discredit, derailing, dominance, stereotyping & objectification, threat of violence, sexual harassment, damning)
• Details of task: Introducing an Arabic Levantine Twitter dataset for Misogynistic language
• Size of dataset: 6,603 direct tweet replies
• Percentage abusive: 48.76%
• Language: Arabic
• Level of annotation: Posts
• Platform: Twitter
• Medium: Text
• Reference: Hala Mulki and Bilal Ghanem. 2021. Let-Mi: An Arabic Levantine Twitter Dataset for Misogynistic Language. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 154–163, Kyiv, Ukraine (Virtual). Association for Computational Linguistics

Are They our Brothers? Analysis and Detection of Religious Hate Speech in the Arabic Twittersphere

• Link to publication: https://ieeexplore.ieee.org/document/8508247
• Link to data: https://github.com/nuhaalbadi/Arabic_hatespeech
• Task description: Binary (Hate, Not)
• Details of task: Religious subcategories
• Size of dataset: 6,136
• Percentage abusive: 0.45
• Language: Arabic
• Level of annotation: Posts
• Platform: Twitter
• Medium: Text
• Reference: Albadi, N., Kurdi, M. and Mishra, S., 2018. Are they Our Brothers? Analysis and Detection of Religious Hate Speech in the Arabic Twittersphere. In: International Conference on Advances in Social Networks Analysis and Mining. Barcelona, Spain: IEEE, pp.69-76.

Multilingual and Multi-Aspect Hate Speech Analysis (Arabic)

• Link to publication: https://arxiv.org/abs/1908.11049
• Link to data: https://github.com/HKUST-KnowComp/MLMA_hate_speech
• Task description: Detailed taxonomy with cross-cutting attributes: Hostility, Directness, Target Attribute, Target Group, How annotators felt on seeing the tweet.
• Details of task: Gender, Sexual orientation, Religion, Disability
• Size of dataset: 3,353
• Percentage abusive: 0.64
• Language: Arabic
• Level of annotation: Posts
• Platform: Twitter
• Medium: Text
• Reference: Ousidhoum, N., Lin, Z., Zhang, H., Song, Y. and Yeung, D., 2019. Multilingual and Multi-Aspect Hate Speech Analysis. ArXiv,.

L-HSAB: A Levantine Twitter Dataset for Hate Speech and Abusive Language

• Link to publication: https://www.aclweb.org/anthology/W19-3512
• Link to data: https://github.com/Hala-Mulki/L-HSAB-First-Arabic-Levantine-HateSpeech-Dataset
• Task description: Ternary (Hate, Abusive, Normal)
• Details of task: Group-directed + Person-directed
• Size of dataset: 5,846
• Percentage abusive: 0.38
• Language: Arabic
• Level of annotation: Posts
• Platform: Twitter
• Medium: Text
• Reference: Mulki, H., Haddad, H., Bechikh, C. and Alshabani, H., 2019. L-HSAB: A Levantine Twitter Dataset for Hate Speech and Abusive Language. In: Proceedings of the Third Workshop on Abusive Language Online. Florence, Italy: Association for Computational Linguistics, pp.111-118.

Abusive Language Detection on Arabic Social Media (Twitter)

• Link to publication: https://www.aclweb.org/anthology/W17-3008

• Link to data: http://alt.qcri.org/~hmubarak/offensive/TweetClassification-Summary.xlsx

• Task description: Ternary (Obscene, Offensive but not obscene, Clean)

• Details of task: Incivility

• Size of dataset: 1,100

• Percentage abusive: 0.59

• Language: Arabic

• Level of annotation: Posts

• Platform: Twitter

• Medium: Text

• Reference: Mubarak, H., Darwish, K. and Magdy, W., 2017. Abusive Language Detection on Arabic Social Media. In: Proceedings of the First Workshop on Abusive Language Online. Vancouver, Canada: Association for Computational Linguistics, pp.52-56.

• Dataset reader: 🤗 strombergnlp/offenseval_2020

Abusive Language Detection on Arabic Social Media (Al Jazeera)

• Link to publication: https://www.aclweb.org/anthology/W17-3008
• Link to data: http://alt.qcri.org/~hmubarak/offensive/AJCommentsClassification-CF.xlsx
• Task description: Ternary (Obscene, Offensive but not obscene, Clean)
• Details of task: Incivility
• Size of dataset: 32,000
• Percentage abusive: 0.81
• Language: Arabic
• Level of annotation: Posts
• Platform: AlJazeera
• Medium: Text
• Reference: Mubarak, H., Darwish, K. and Magdy, W., 2017. Abusive Language Detection on Arabic Social Media. In: Proceedings of the First Workshop on Abusive Language Online. Vancouver, Canada: Association for Computational Linguistics, pp.52-56.

Dataset Construction for the Detection of Anti-Social Behaviour in Online Communication in Arabic

• Link to publication: https://www.sciencedirect.com/science/article/pii/S1877050918321756
• Link to data: https://onedrive.live.com/?authkey=!ACDXj_ZNcZPqzy0&id=6EF6951FBF8217F9!105&cid=6EF6951FBF8217F9
• Task description: Binary (Offensive, Not)
• Details of task: Incivility
• Size of dataset: 15,050
• Percentage abusive: 0.39
• Language: Arabic
• Level of annotation: Posts
• Platform: YouTube
• Medium: Text
• Reference: Alakrot, A., Murray, L. and Nikolov, N., 2018. Dataset Construction for the Detection of Anti-Social Behaviour in Online Communication in Arabic. Procedia Computer Science, 142, pp.174-181.

<a id="Bengali-header"></a>

Bengali

Hate Speech Detection in the Bengali language: A Dataset and its Baseline Evaluation

• Link to publication: https://arxiv.org/pdf/2012.09686.pdf
• Link to data: https://www.kaggle.com/naurosromim/bengali-hate-speech-dataset
• Task description: Binary (hateful, not)
• Details of task: Several categories: sports, entertainment, crime, religion, politics, celebrity and meme
• Size of dataset: 30,000
• Percentage abusive: 0.33
• Language: Bengali
• Level of annotation: Posts
• Platform: Youtube and Facebook
• Medium: Text
• Reference: Romim, N., Ahmed, M., Talukder, H., & Islam, M. S. (2021). Hate speech detection in the bengali language: A dataset and its baseline evaluation. In Proceedings of International Joint Conference on Advances in Computational Intelligence (pp. 457-468). Springer, Singapore.

<a id="Chinese-header"></a>

Chinese

SWSR: A Chinese Dataset and Lexicon for Online Sexism Detection

• Link to publication: https://www.sciencedirect.com/science/article/abs/pii/S2468696421000604#fn1
• Link to data: https://doi.org/10.5281/zenodo.4773875
• Task description: Binary (Sexist, Non-sexist), Categories of sexism (Stereotype based on Appearance, Stereotype based on Cultural Background, MicroAggression, and Sexual Offense), Target of sexism (Individual or Generic)
• Details of task: Sexism detection on social media in Chinese
• Size of dataset: 8,969 comments from 1,527 weibos
• Percentage abusive: 34.5%
• Language: Chinese
• Level of annotation: Posts
• Platform: Sina Weibo
• Medium: Text
• Reference: Aiqi Jiang, Xiaohan Yang, Yang Liu, Arkaitz Zubiaga, SWSR: A Chinese dataset and lexicon for online sexism detection, Online Social Networks and Media, Volume 27, 2022, 100182, ISSN 2468-6964.

<a id="Croatian-header"></a>

Croatian

CoRAL: a Context-aware Croatian Abusive Language Dataset

• Link to publication: https://aclanthology.org/2022.findings-aacl.21/
• Link to data: https://github.com/shekharRavi/CoRAL-dataset-Findings-of-the-ACL-AACL-IJCNLP-2022
• Task description: Multi-class based on context dependency categories (CDC)
• Details of task: Detectioning CDC from abusive comments
• Size of dataset: 2,240
• Percentage abusive: 100%
• Language: Croatian
• Level of annotation: Posts
• Platform: Newspaper comments
• Medium: Text