CRNLP_05_TPandSA.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3e6572f4",
   "metadata": {},
   "source": [
    "# NLP V: Introduction to Topic Modeling and Sentiment Analytics\n",
    "\n",
    "In this notebook you'll find the information needed to classify texts and to detect polarization in texts.\n",
    "\n",
    "## Topic Modeling\n",
    "\n",
    "In this Jupyter Notebook, we will delve into the realms of topic modeling and conversational agents. Our aim is to explore how algorithms can be employed to automatically classify sentences into various topics, which can be particularly useful for topic moderation among other applications.\n",
    "\n",
    "### Data Preprocessing Overview\n",
    "\n",
    "First, we start by converting a set of sentences into a matrix. In this matrix, each column represents a unique word, and each row corresponds to a sentence. We utilize 1s and 0s to indicate whether or not a particular word is present in a given sentence.\n",
    "\n",
    "Let us start importing libraries, what else can we do? 👥"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "efae339c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import nltk, string\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer, CountVectorizer\n",
    "from sklearn.decomposition import NMF, LatentDirichletAllocation\n",
    "import numpy as np\n",
    "\n",
    "# Do't forget to import your library; you will need it.\n",
    "# Type your code here:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9ae5334",
   "metadata": {},
   "source": [
    "### Load the corpus texts\n",
    "\n",
    "The corpus is similar to what we use for tagging; it consists of a set of sentences."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "44e30320",
   "metadata": {},
   "outputs": [],
   "source": [
    "trainCorpus = [\"Me gustan las vacas\",\n",
    "               \"Me gustan los caballos\",\n",
    "               \"odio los perros\",\n",
    "               \"odio los caballos\",\n",
    "               \"me gustan las ranas\",\n",
    "               \"me gusta el helado\",\n",
    "               \"no quiero comer\",\n",
    "               \"los helados son cremosos\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27733807",
   "metadata": {},
   "source": [
    "### Vectorised the corpus texts\n",
    "\n",
    "The next step is to vectorize the sentences; i.e. the corpus texts. This means to convert the text in a vector of frequencies."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "701a53c5",
   "metadata": {},
   "outputs": [],
   "source": [
    "puncts = [c for c in string.punctuation]\n",
    "spanish_stopwords = nltk.corpus.stopwords.words('spanish') + puncts\n",
    "vectorizer  = CountVectorizer(stop_words=spanish_stopwords)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f17fa854",
   "metadata": {},
   "source": [
    "Stopwords, together with punctuation signs, are useful to have them at hand for cleaning our dataset of undesired elements that do not add value. They usually consist of commonly occurring words like 'the', 'and', 'is', as well as punctuation marks such as commas, periods, and exclamation marks, which may not be meaningful in the analysis.\n",
    "\n",
    "**Reminder:** We're implementing this cleaning step because we are operating within a free context schema, where we're not restricted by any specific field or subject matter for our text data. In such a case, removing these stopwords and punctuation signs helps us focus on the words that actually carry significance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a328992c",
   "metadata": {},
   "outputs": [],
   "source": [
    "vectorizer.fit(trainCorpus)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b67184c7",
   "metadata": {},
   "source": [
    "As you can see, it creates an array to filter out what is truly significant. \n",
    "\n",
    "What it has actually learned is the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fbea745b",
   "metadata": {},
   "outputs": [],
   "source": [
    "tf_feature_names = vectorizer.get_feature_names_out()\n",
    "print('Atributos:', vectorizer.get_feature_names_out())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cedc652f",
   "metadata": {},
   "source": [
    "We transform the documents into a document-term matrix (often abbreviated as DTM) using topic frequency (TF).\n",
    "\n",
    "**Little Excercice:** Print the shape of the `tfMatrix` matrix. Do you andestand what each dimension means?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "40620cad",
   "metadata": {},
   "outputs": [],
   "source": [
    "tfMatrix = vectorizer.transform(trainCorpus)\n",
    "# print(tfMatrix)\n",
    "\n",
    "print('Matrix:\\n', tfMatrix.toarray())\n",
    "\n",
    "# Type your code here:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "699b300b",
   "metadata": {},
   "source": [
    "**Carefully examine the matrix and compare it with both: `tf_feature_names` and the sentences in the corpus.** \n",
    "\n",
    "The `tfMatrix` is our dataset, where:\n",
    "- each row represents a sample (a document in the corpus).\n",
    "- each column represents an attribute (the frequency of a word in that document).\n",
    "\n",
    "**Room for Improvement?**\n",
    "\n",
    "Absolutely! You could implement stemming to group similar word forms together. For example, this would allow the algorithm to recognize 'gustan' as similar to 'gusta', or 'helados' as similar to 'helado', and so on. To keep this notebook from becoming overly lengthy, let's move on to some more key concepts."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65b11429",
   "metadata": {},
   "source": [
    "### The LDA algorithm for extracting topics\n",
    "\n",
    "The next algorithm we'll explore allows us to identify the distinct characteristics of various themes or topics.\n",
    "\n",
    "We'll use the **Latent Dirichlet Allocation (LDA)** model for topic modeling. Here's a short explanation.\n",
    "\n",
    "LDA is a type of probabilistic model used for topic modeling. Topic modeling is the task of automatically identifying topics present in a text corpus. The idea is to uncover the hidden thematic structure in a large collection of documents.\n",
    "\n",
    "**How Does LDA Work?**\n",
    "\n",
    "1) **Initialization:** You specify the number of topics you want the model to find in your corpus. For example, in our case, the corpus seems to clearly revolve around \"love\" and \"hate\" as one of the topics. Other parameters control the learning process.\n",
    "2) **Random Assignment**: Each word in each document is initially assigned to a random topic.\n",
    "3) **Iterative Refinement:** The model goes through multiple iterations, reassigning each word to a topic based on:\n",
    "    - How often the topic occurs in the document.\n",
    "    - How often the word occurs in the topic across all documents.\n",
    "    \n",
    "    You can change this parameter if desired.\n",
    "4) **Convergence:** The algorithm iteratively refines these assignments until they converge to a stable state or after a fixed number of iterations.\n",
    "5) **Output:** The end result is a set of topics, each represented as a collection of words, and the weight or probability of each word belonging to a given topic."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ba26cce4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the number of topics and initialize the Latent Dirichlet Allocation model\n",
    "topics = 2\n",
    "lda_model = LatentDirichletAllocation(\n",
    "    n_components=topics,      # Number of topics\n",
    "    max_iter=5,               # Maximum number of iterations for the optimization\n",
    "    learning_method='online', # Learning method, online variational Bayes\n",
    "    learning_offset=50.,      # A parameter to downweigh early iterations in online learning\n",
    "    random_state=0            # Random seed\n",
    ").fit(tfMatrix)               # Fit the model to the term-frequency matrix\n",
    "\n",
    "# The H matrix contains the words associated with each topic\n",
    "# The components_ attribute gives us the topics found\n",
    "H = lda_model.components_\n",
    "\n",
    "# The W matrix contains the topic distribution for each document\n",
    "# The transform method gives us the topic distribution for each document\n",
    "W = lda_model.transform(tfMatrix)\n",
    "\n",
    "# Define the number of top words to display for each topic and the number of top documents for each topic, respectively\n",
    "n_top_words = 2\n",
    "n_top_documents = 3\n",
    "\n",
    "# Function to display the topics\n",
    "def display_topics(H, W, feature_names, documents, n_top_words, n_top_documents):\n",
    "    for topic_idx, topic in enumerate(H):\n",
    "        print('-------------')\n",
    "        print('Topic', topic_idx)\n",
    "        # Print the top words in the topic\n",
    "        for i in topic.argsort()[:-n_top_words - 1:-1]:\n",
    "            print(feature_names[i])\n",
    "        # Print the documents that have the highest probability for this topic\n",
    "        top_doc_indices = np.argsort(W[:, topic_idx])[::-1][0:n_top_documents]\n",
    "        for doc_index in top_doc_indices:\n",
    "            print(trainCorpus[doc_index])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bbcdf2c4",
   "metadata": {},
   "outputs": [],
   "source": [
    "display_topics(H, W, tf_feature_names, trainCorpus, n_top_words, n_top_documents)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69e03d9f",
   "metadata": {},
   "source": [
    "It's worth mentioning that since I set the algorithm to return strictly three documents for each topic, and there are only two documents that closely match these topics, the algorithm returns 'no quiero comer' as one of the top documents."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d98a174",
   "metadata": {},
   "source": [
    "## Sentiment Analysis using NLTK's Vader\n",
    "\n",
    "Sentiment Analysis is the computational study of people's opinions, sentiments, emotions, and attitudes. This technique is commonly applied to customer reviews, social media comments, and survey responses to make data-driven decisions. In the field of Natural Language Processing (NLP), it holds a significant place.\n",
    "\n",
    "In this notebook, we will utilize Vader (Valence Aware Dictionary and sEntiment Reasoner), a pre-built sentiment analysis tool that is a part of the Natural Language Toolkit (NLTK). Vader is particularly good at handling social media text, short sentences, and even emoticons. Of course, manual sentiment analysis is an option, but given that we've already invested significant effort into manual methods, it's both practical and efficient to leverage Vader for this task.\n",
    "\n",
    "The goal is to train a model to classify sentences or text into various sentiment categories—positive, negative, or neutral.\n",
    "\n",
    "### How Vader Works\n",
    "\n",
    "Vader works by tokenizing the text into individual words (tokens). It then checks these tokens against its predefined list of positive, negative, and neutral words. Each word is assigned a score, and the overall sentiment of the sentence is calculated based on the sum of these scores.\n",
    "\n",
    "Importing NLTK Libraries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a763face",
   "metadata": {},
   "outputs": [],
   "source": [
    "from nltk.sentiment.vader import SentimentIntensityAnalyzer"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c776704",
   "metadata": {},
   "source": [
    "### Corpus for Analysis\n",
    "\n",
    "In the corpus that we've prepared for analysis, we've included a variety of sentences with differing levels of complexity. These range from simple statements to more nuanced expressions, in order to test the capabilities of our sentiment analysis model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a94b2b42",
   "metadata": {},
   "outputs": [],
   "source": [
    "corpus = [\"VADER is smart, handsome, and funny.\", # example of a positive sentence\n",
    "\"VADER is smart, handsome, and funny!\", # Detection of exclamatory emphasis (increased intensity of feeling).\n",
    "\"VADER is very smart, handsome, and funny.\", # detection of augmentative words (increased intensity)\n",
    "\"VADER is VERY SMART, handsome, and FUNNY.\", # emphasis derived from capitalization\n",
    "\"VADER is VERY SMART, handsome, and FUNNY!!!\",# combination of the above\n",
    "\"VADER is VERY SMART, really handsome, and INCREDIBLY FUNNY!!!\",# combination of the above to the highest level\n",
    "\"The book was good.\", # positive sentence\n",
    "\"The book was kind of good.\", # decrease of positivity (intensity adjustment)\n",
    "\"The plot was good, but the characters are uncompelling and the dialog is not great.\", # negation\n",
    "\"A really bad, horrible book.\", # negative sentence with intensity enhancers\n",
    "\"At least it isn't a horrible book.\", # negation of negativity\n",
    "\":) and :D\", # emoticons\n",
    "\"\", # empty strings are treated correctly\n",
    "\"Today kinda sux\", # slang word detection\n",
    "\"Today KINDA SUX!\", # combination of slang with capitalisation and exclamation (increases sentiment)\n",
    "\"I'll get by\", # this sentence is neutral\n",
    "\"Today kinda sux, think I'll get by\", # This example serves to compare with the following sentence    \n",
    "\"Today kinda sux, but I'll get by\" # 'but' softens the negativity of the previous sentence.\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7c2581f",
   "metadata": {},
   "source": [
    "### Execute Pre-Built Function\n",
    "\n",
    "We will make use of the `SentimentIntensityAnalyzer()` function from the VADER. This function will help us quickly and efficiently determine the sentiment of each sentence in our corpus."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "da6c8cbb",
   "metadata": {},
   "outputs": [],
   "source": [
    "sid = SentimentIntensityAnalyzer()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72bbaa5e",
   "metadata": {},
   "source": [
    "Now, let's analyze the sentences in our example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ba5758ea",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Iterate through each sentence in the corpus\n",
    "for sentence in corpus:\n",
    "\n",
    "    # Print the sentence that is about to be analyzed\n",
    "    print(sentence)\n",
    "    \n",
    "    # Calculate the sentiment scores of the sentence and store them in a dictionary 'ss'\n",
    "    ss = sid.polarity_scores(sentence)\n",
    "    \n",
    "    # Sort and print each component of the sentiment score\n",
    "    # 'compound' is the aggregated sentiment score\n",
    "    # 'neg' represents negativity\n",
    "    # 'neu' represents neutrality\n",
    "    # 'pos' represents positivity\n",
    "    for k in sorted(ss):\n",
    "        print(f\"{k}: {ss[k]}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "243ffc59",
   "metadata": {},
   "source": [
    "### Handmade Sentiment Analyzer: A Custom Approach\n",
    "\n",
    "In the following section, we will create our own Sentiment Analyzer. We will choose a custom set of words that can be associated with a positive or a negative sentiment. We'll also associate these custom words to sentiment scores.\n",
    "\n",
    "**Context Matters**\n",
    "\n",
    "The context in which you use this model is crucial. For instance, in a restaurant review, the term 'chorizo' is likely neutral or even positive. However, in the context of a Twitter post discussing corruption, 'chorizo' could be decidedly negative.\n",
    "\n",
    "**The Big Picture**\n",
    "\n",
    "The overarching goal of this approach is to sift through a large set of data and categorize it into different thematic clusters or sentiment groups. We define one set with positive sentiments, another set with negative sentiments, and perhaps another set for neutral sentiments.\n",
    "\n",
    "The algorithm learns these categorizations during its training phase and applies this knowledge to unseen data. The quality of its predictions is directly proportional to the size and quality of the training data. This versatile model can be adapted for a wide range of applications—from predicting the mood of a song to determining the sentiment conveyed in a photograph.\n",
    "\n",
    "So, whether you're employing VADER or a more advanced sentiment analyzer like BERT, the customizable approach outlined below provides invaluable insights. Faced with something complex? Simply break it down into manageable and understandable components."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cf98512a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Predefined positive and negative words\n",
    "p_pos = ['inteligente', 'simpático', 'bueno', 'amable']\n",
    "p_neg = ['malo', 'feo', 'ladrón', 'corrupto', 'chorizo']\n",
    "\n",
    "def sentiment(_frase):\n",
    "\n",
    "    # Tokenization and cleaning the sentence using predefined functions from your library\n",
    "    tokens = tokenization.tokenize(_frase)\n",
    "    tokens = tokenization.clean_sw(tokens)\n",
    "\n",
    "    # Initializing counters for positive, neutral and negative words\n",
    "    toks = len(tokens)\n",
    "    pos = 0\n",
    "    neu = 0\n",
    "    neg = 0\n",
    "\n",
    "    # Loop through the tokens and increment counters based on the word sentiment\n",
    "    for token in tokens:\n",
    "        if token in p_pos:\n",
    "            pos += 1\n",
    "        elif token in p_neg:\n",
    "            neg += 1\n",
    "        else:\n",
    "            neu += 1\n",
    "    \n",
    "    # Applying weights to positive and negative counts to get a polarity score\n",
    "    pol = (pos * 0.6) - (neg * 0.7)\n",
    "    neu *= 0.1  # Assign some weight to neutral words\n",
    "\n",
    "    # Calculate the final sentiment score\n",
    "    res = pol + neu\n",
    "\n",
    "    # Display the results\n",
    "    print('Sentence:', _frase)\n",
    "    print('--------------------')\n",
    "    print('Sentiment:', res)\n",
    "    print('Positive:', pos)\n",
    "    print('Neutral:', neu)\n",
    "    print('Negative:', neg)\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9826c28f",
   "metadata": {},
   "outputs": [],
   "source": [
    "sentiment('Mortadelo es un político feo pero malo')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a55a9a23",
   "metadata": {},
   "source": [
    "The modifications are endless! Building upon our original custom sentiment analyzer, we introduce an innovative feature—dynamic weighting. This added complexity enables a more nuanced understanding of sentiment in the provided text.\n",
    "\n",
    "In the original version, each positive and negative word received a static weight, potentially leading to imprecise sentiment scoring. However, with dynamic weighting, the significance of each positive or negative word is influenced by its frequency relative to the total number of tokens in the sentence."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b18ed3c5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Predefined positive and negative words\n",
    "p_pos = ['inteligente', 'simpático', 'carismático', 'amable']\n",
    "p_neg = ['extinguiremos', 'megaminería', 'cambio climático', 'corrupto', 'lamentablemente', 'choreó', 'terroríficas']\n",
    "\n",
    "def sentiment(_frase):\n",
    "    tokens = tokenization.tokenize(_frase)\n",
    "    tokens = tokenization.clean_sw(tokens)\n",
    "    \n",
    "    # Initialize counters\n",
    "    pos = 0\n",
    "    neu = 0\n",
    "    neg = 0\n",
    "    \n",
    "    # Total number of tokens in the sentence\n",
    "    total_tokens = len(tokens)\n",
    "    \n",
    "    # Increment counters based on the word sentiment\n",
    "    for token in tokens:\n",
    "        if token in p_pos:\n",
    "            pos += 1\n",
    "        elif token in p_neg:\n",
    "            neg += 1\n",
    "        else:\n",
    "            neu += 1\n",
    "    \n",
    "    # Calculate the proportion of positive and negative tokens\n",
    "    if total_tokens == 0:  # To avoid division by zero\n",
    "        prop_pos = 0\n",
    "        prop_neg = 0\n",
    "    else:\n",
    "        prop_pos = pos / total_tokens\n",
    "        prop_neg = neg / total_tokens\n",
    "    \n",
    "    # Dynamic weighting based on the proportion of positive and negative tokens\n",
    "    weight_pos = 0.6 + prop_pos\n",
    "    weight_neg = 0.4 + prop_neg\n",
    "    \n",
    "    # Calculating polarity score with dynamic weights\n",
    "    pol = (pos * weight_pos) - (neg * weight_neg)\n",
    "\n",
    "    # Apply some weight to neutral words\n",
    "    neu_score = neu * 0.1 \n",
    "    \n",
    "    # Calculate the final sentiment score\n",
    "    res = pol + neu_score\n",
    "    \n",
    "    # Display the results\n",
    "    print('Sentence:', _frase)\n",
    "    print('---------------------------------------')\n",
    "    print('Positive Words:', pos)\n",
    "    print('Neutral Words:', neu)\n",
    "    print('Negative Words:', neg)\n",
    "    print('Sentiment Score:', res)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a3bf4375",
   "metadata": {},
   "outputs": [],
   "source": [
    "sentiment('Salchichón se choreó la partida presupuestaria de la obra pública. Sin embargo, como es un político carismático ganará la elecciones. Todos estamos al tanto de sus terroríficas políticas en post de la megaminería, que contribuyen al cambio climático. Lamentablemente, nos extinguiremos como raza.')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22e070d9",
   "metadata": {},
   "source": [
    "**Final Remarks: The Power of Creative Combinations**\n",
    "\n",
    "As we close this tutorial, it's crucial to note that the tools and techniques we've explored are not isolated entities but components of a greater analytical framework. The true power of Natural Language Processing and Sentiment Analysis lies in creatively combining these elements to tailor solutions to specific problems or datasets.\n",
    "\n",
    "Whether it's enhancing a simple Sentiment Analyzer with dynamic weighting, incorporating Latent Dirichlet Allocation for topic modeling, or even merging different machine learning models—your creativity is the limit.\n",
    "\n",
    "Each method complements the others, and together they can provide more nuanced and accurate insights into textual data than any single approach could achieve on its own. Feel free to experiment, innovate, and most importantly, have fun exploring the endless possibilities that these tools offer."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}