Note that this page is outdated and we have an updated version of the corpus.

Constructiveness and Toxicity Corpus - CTC

The CSV file contains annotations for constructiveness and toxicity in news comments.

Paper

This work is described in a paper to appear in the Proceedings of the First Workshop on Abusive Language Online, at the Meeting of the Association for Computational Linguistics in Vancouver, August 2017. https://sites.google.com/site/abusivelanguageworkshop2017/

Kolhatkar, Varada and Maite Taboada (2017) Constructive language in news comments. Proceedings of the First Workshop on Abusive Language Online, Association for Computational Linguistics. Vancouver, August 2017.

Data

A collection of 1,121 comments posted in response to opinion articles, on the website of the newspaper The Globe and Mail (https://www.theglobeandmail.com/). Comments were responses to 10 different articles covering a variety of subjects: technology, immigration, terrorism, politics, budget, social issues, religion, property, and refugees. For half of the articles, we included only top-level comments. For the other half, we included both top-level comments and responses.

Annotations

We used CrowdFlower (https://www.crowdflower.com/) as our crowdsourcing annotation platform and annotated the comments for constructiveness. We asked the annotators to first read the articles, and then to tell us whether the displayed comment was constructive or not.

For toxicity, we asked annotators a multiple-choice question, How toxic is the comment? Four answers were possible:

• Very toxic
• Toxic
• Mildly toxic
• Not toxic

More information on the annotation, and the instructions to annotators, is available in the CrowdFlower_instructions file.