A collection of corpora for named entity recognition (NER) and entity recognition tasks. These annotated datasets cover a variety of languages, domains and entity types.

NLTK Data

Data repository for pretrained NLP models and NLP corpora.

微信公众号语料库

Fuzzing resources for feeding various fuzzers with input. 🔧

A web-based engine for creating and annotating textual corpora

A collaborative catalog of resources for Indian language NLP

Links to Russian corpora + Python functions for loading and parsing

The Self-dialogue Corpus - a collection of self-dialogues across music, movies and sports

Automatic categorization of documents, consists in assigning a category to a text based on the information it contains. We'll follow different approach of Supervised Machine Learning.

An R package for dynamic exploration of text collections

Named Entity Recognition for biomedical entities

An advanced, extensible web front-end for the Manatee-open corpus search engine

Unannotated Spanish 3 Billion Words Corpora

A Django App for gathering written and spoken corpora for indigenous languages.

Tools for filtering and cleaning parallel and monolingual corpora

A variety of loaders for various NLP corpora.

Reading the data from OPIEC - an Open Information Extraction corpus

Open Korean NLP Dataset Curation for the Users All Around the Globe

An unofficial Python API that allows users to create a corpus of lyrical text from their favorite artists and billboard charts

Table compiling the list of biomedically-related corpora available for named entity recognition (and some also suitable for association detection). This has been published as part of the paper: Dieter Galea, Ivan Laponogov, Kirill Veselkov; Exploiting and assessing multi-source data for supervised biomedical named entity recognition, Bioinformatics, bty152, https://doi.org/10.1093/bioinformatics/bty152 . If you would like to add other (or your) corpora, please submit a pull request and I'll happily approve it.

Dataset containing Semantic Relations and Metadata, for Training and Evaluating Distributional Semantic Models in English and Mandarin Chinese

Textstelle is a collection of corpora for the creation of bots and other things that generate text 🤖

Clean corpus generic script made with tm package

Extract corpora from Wikipedia dumps

Simple CORPORA list crawler

The Data Format for Digital Linguistics (DaFoDiL)

A curated list of resources dedicated to Natural Language Processing (NLP) of Cantonese | 粵語 NLP 資源

repo for Tibetan corpora

The Potsdam Twitter Sentiment Corpus