Data science - text analysis (R).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data science - text analysis\n",
    "\n",
    "Data science tools allow classifying data into groups, either using examples to support in classification (_supervised_ machine learning) or usign statistical analysis on word-level similarities (_unsupervised_ machine learning).\n",
    "\n",
    "To get a grasp of these methods, see SICSS videos on [dictionary-based methods](https://www.youtube.com/watch?v=wSIi2ZRKjaE) and [topic models](https://www.youtube.com/watch?v=IUAHUEy1V0Q).\n",
    "\n",
    "We learn how to\n",
    "1. read and transform text data into something computers can work on (using [quanteda](https://quanteda.io/))\n",
    "1. conduct a supervised machine learning task on the data (i.e., teaching from examples)\n",
    "1. conduct an unsupervised machine learning task on the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading data with quanteda\n",
    "\n",
    "Quanteda is a popular library for computational text analysis, supporing various pre-processing and data management tasks common for text-based data.\n",
    "We first need to transform the textual material as a _corpus_ to make workable as a computational data source.\n",
    "After this we extract the document-term matrix from the data, ensuring that common English stopwords are excluded and words are stemmed before further analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data <- read.csv('./data/ExtractedTweets.csv', stringsAsFactors = FALSE )\n",
    "data <- data[ sample(1:nrow(data), size = 500, replace=FALSE), ] ## dataset is fairly large, let's work with a smaller dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "library(\"quanteda\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.corpus <- corpus( data, text_field = \"Tweet\" )\n",
    "data.tokens <- tokens( data.corpus, remove_punct = TRUE, remove_symbols = TRUE, remove_numbers = TRUE )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "documenttermmatrix <- dfm( data.tokens )\n",
    "documenttermmatrix <- dfm_remove( documenttermmatrix, pattern = stopwords(\"english\") ) ## remove common english words which often do not help in analysis\n",
    "documenttermmatrix <- dfm_wordstem( documenttermmatrix )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hist( featfreq( documenttermmatrix ) ) ## most words are used just a few times (x-axis: how many times userd, y-axis: frequenxcy in the class)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Supervised machine learning\n",
    "\n",
    "The gist of supervised machine learning is teaching computers to classify based on examples:\n",
    "for example, showing the computer several examples of Republican tweets and Democratic tweets should allow computer to classify previously unseen tweet into either parties.\n",
    "\n",
    "`quanteda.textmodels` is a [package](https://cran.r-project.org/web/packages/quanteda.textmodels/quanteda.textmodels.pdf) spesically tuned for doing text classifications and is fairly easy to use, with a helpful [tutorial] as well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "library(quanteda.textmodels)\n",
    "library(caret)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class <- docvars( documenttermmatrix )$Party\n",
    "table( class )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.nb <- textmodel_nb(documenttermmatrix, class )\n",
    "summary( model.nb )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Accuracy\n",
    "\n",
    "All accuracy-based evaluations focus on comparing actual classifications we know are correct with predicted classifications.\n",
    "This allows us to computer accuracy %, Kappa-values as well as confusion matrixes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "actual_class <- class\n",
    "predicted_class <- predict( model.nb , newdata = documenttermmatrix )\n",
    "tab_class <- table(actual_class, predicted_class)\n",
    "\n",
    "confusionMatrix(tab_class, mode = \"everything\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercises\n",
    "\n",
    "* Increase the sample to 5000 Tweets and redo the analysis\n",
    "* Instead of using the sampled tweets to compute the accuracy, choose another 2500 Tweets from the data and compute accuracy metrics with them. (Thus, you have now taken two different samples from the data, one for _training_ the data and another for _testing_ or validating how well the trained dataset works with unseen data. See [Wikipedia](https://en.wikipedia.org/wiki/Training,_validation,_and_test_sets) for detailed information.)\n",
    "* Try out other machine learning method, support vector machines. Which one produces higher quality of results?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Unsupervised machine learning\n",
    "\n",
    "Unlike in supervised machine learning, in unsupervised machine learning the classification groups emerge from statistical analysis based on data.\n",
    "There are various different ways to achive this, ranging from latent semantical scaling and $k$-means to topic models.\n",
    "Each of them produces somewhat different results and at least I am not aware of any best practices on how to choose which method to use.\n",
    "\n",
    "Next we look at topic models using `stm` package ([documentation](https://rdocumentation.org/packages/stm/), [example](https://github.com/bstewart/stm/blob/master/vignettes/stmVignette.pdf?raw=true)).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## load data\n",
    "\n",
    "library(quanteda)\n",
    "\n",
    "data <- read.csv('./data/ExtractedTweets.csv', stringsAsFactors = FALSE )\n",
    "data <- data[ sample(1:nrow(data), size = 500, replace=FALSE), ] ## dataset is fairly large, let's work with a smaller dataset  \n",
    "\n",
    "data.corpus <- corpus( data, text_field = \"Tweet\" )\n",
    "data.tokens <- tokens( data.corpus, remove_punct = TRUE, remove_symbols = TRUE, remove_numbers = TRUE )\n",
    "\n",
    "documenttermmatrix <- dfm( data.tokens )\n",
    "documenttermmatrix <- dfm_remove( documenttermmatrix, pattern = stopwords(\"english\") ) ## remove common english words which often do not help in analysis\n",
    "documenttermmatrix <- dfm_wordstem( documenttermmatrix )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "library(stm)\n",
    "stm.data  <- convert( documenttermmatrix , to = \"stm\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.stm <- stm( stm.data$documents, stm.data$vocab, K = 5, data = stm.data$meta )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary( model.stm )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercises\n",
    "* Vary the number of topics (parameter `K`in the code above) and re-run the analysis. What kind of differences can you detect?\n",
    "* Are you happy with words used in the analysis? Remove additional words if nececcarily.\n",
    "* From the vignette, look how to do an effect plot to identify how Republicans and Democrats differ across all topics. Are there topics which have a clear difference?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "4.2.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}