project-3/wordnet.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# A WordNet Experience\n",
        "WordNet is a Princeton project grouping English words in sets linked by semantics or lexical relationships. Words are often linked because they are synonyms into sets called synsets, which reveals a hierarchical structures in English. The database is useful, along with databases like nltk's text, to examine relationships between words. I am here to explore it!\n",
        "\n",
        "## The Chosen Noun\n",
        "For demonstration purposes, I'll select a noun to examine it's relationships, and it's synset. Lets start by printing as much information as we can about the noun, and traveling up the *hiearchy* that WordNet has found words form."
      ],
      "metadata": {
        "id": "KRdz3sjLhMAT"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 72,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lL3wxSCuhLBg",
        "outputId": "41988a62-fbb4-4eb0-ea8c-ccd7d13a6492"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[Synset('umbrella.n.01'), Synset('umbrella.n.02'), Synset('umbrella.n.03'), Synset('umbrella.s.01')]\n"
          ]
        }
      ],
      "source": [
        "# Has nltk never been set up in this environment? Run these lines:\n",
        "# (Here is a github issue about why omw-1.4 is seperate)\n",
        "# https://github.com/nltk/nltk/issues/3024\n",
        "import nltk\n",
        "nltk.download(\"wordnet\")\n",
        "nltk.download(\"omw-1.4\")\n",
        "\n",
        "from nltk.corpus import wordnet as wn\n",
        "\n",
        "# Now... what noun to use.... Asking ChatGPT for a random word, I got back:\n",
        "# \"Sure, here's a random noun: \"umbrella\"\"\n",
        "the_chosen_noun = \"umbrella\"\n",
        "\n",
        "# Lets See it's synsets\n",
        "synsets = wn.synsets(the_chosen_noun)\n",
        "print(synsets)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can see that there are 4 different meanings for the word umbrella. I must admit I don't even know what the 's' means in the fourth meaning. Thats why we explore!"
      ],
      "metadata": {
        "id": "41cU2AV8kzVx"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "for synset in synsets:\n",
        "  print(synset.name() + \": \" + synset.definition())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HrY_fTVak-jU",
        "outputId": "b4899100-8e65-4ea3-f4b1-803ea5e34ba5"
      },
      "execution_count": 73,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "umbrella.n.01: a lightweight handheld collapsible canopy\n",
            "umbrella.n.02: a formation of military planes maintained over ground operations or targets\n",
            "umbrella.n.03: having the function of uniting a group of similar things\n",
            "umbrella.s.01: covering or applying simultaneously to a number of similar items or elements or groups\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Well, those are the definitions you would expect, but I still couldn't quite tell what 's' meant. I asked chatGPT again!\n",
        "\n",
        "> In WordNet, the letter \"s\" in a synset name indicates that the synset represents an adjective satellite. An adjective satellite is a type of adjective that modifies the meaning of a noun by specifying a particular aspect or quality of it, rather than directly describing a property of the noun itself. Adjective satellites are often used to convey subtle shades of meaning or to express complex relationships between objects.\n",
        "\n",
        "Considering we were tryinig to get down to the bottom of explaining what an adjective satellite was in class, this is quite a good example! If you had to think of a strange use case of umbrella, what would you think of? \"Umbrella term\" makes the noun *term* encompass a wide range of concepts or ideas. This particular example is fun, as umbrella is a noun in any other case!\n",
        "\n",
        "For fun, lets compare umbrella's use as a noun and adjective satelite when it shares a similar meaning.\n",
        "\n"
      ],
      "metadata": {
        "id": "tJiNmMasnA5-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "for synset in synsets:\n",
        "  print(synset.examples())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HrD2-106nR-7",
        "outputId": "6cc3adbe-26be-478c-fa2f-7e95def1a78e"
      },
      "execution_count": 74,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[]\n",
            "['an air umbrella over England']\n",
            "['the Democratic Party is an umbrella for many liberal groups', 'under the umbrella of capitalism']\n",
            "['an umbrella organization', 'umbrella insurance coverage']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Look at that last example! You can tell that the noun organization has been modified to mean \"An organization that covers simultaneously to a number of similar items\" while no property of the noun was modified (We didn't say the size of the organization was large, we specified particular aspect that was before subtle). Compare that to the noun usage example above it, where umbrella was applied to the meaning of Democratic Party, but by simply saying it was an umbrella (noun).\n",
        "\n",
        "All that nuance is fun, but lets pick the simple handheld collapsible canopy for our *chosen one* in the rest of this exercise. "
      ],
      "metadata": {
        "id": "f8c6i62EqBo9"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "the_chosen_noun = synsets[0]\n",
        "\n",
        "def examine_synset(synset):\n",
        "  # Lets print all we can about umbrellas!\n",
        "  print(\"Definition: \" + synset.definition())\n",
        "\n",
        "  # Given that examples can be empty (and is), we should write a print statement\n",
        "  # that can handle any length of list\n",
        "  examples = synset.examples()\n",
        "  if len(examples):\n",
        "    separator = \", \"\n",
        "    formatted_examples = separator.join(synset.examples())\n",
        "    print(\"Examples: \" + formatted_examples)\n",
        "  else:\n",
        "    print(\"Examples: N/A\")\n",
        "\n",
        "  # Same case for lemmas, we may not have lemmas\n",
        "  lemmas = synset.lemmas()\n",
        "  if len(lemmas):\n",
        "    separator = \", \"\n",
        "    formatted_lemmas = separator.join([lemma.name() for lemma in lemmas])\n",
        "    print(\"Lemmas: \" + formatted_lemmas)\n",
        "  else:\n",
        "    print(\"Examples: N/A\")\n",
        "\n",
        "examine_synset(the_chosen_noun)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Qg-8_T5Ip0OL",
        "outputId": "18368141-e7d2-42cc-a125-c577e4be1dd5"
      },
      "execution_count": 75,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Definition: a lightweight handheld collapsible canopy\n",
            "Examples: N/A\n",
            "Lemmas: umbrella\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Knowing that about our noun, lets traverse the hiearchy by travelling up the 'hypernyms' of the word (or the list of synsets that represent more general concepts that the current sysnset is part of)."
      ],
      "metadata": {
        "id": "Ry637y4qwDXu"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# If a word has multiple hypernyms, we will select the first hypernym\n",
        "# Select a starting hypernym, assuming there is one (mimic a do-while loop)\n",
        "# then just keep printing the next hypernym until there are none left\n",
        "def traverse(synset) :\n",
        "  hyper = synset.hypernyms()[0]\n",
        "  while hyper:\n",
        "    print(hyper)\n",
        "    if not hyper.hypernyms():\n",
        "      break\n",
        "    hyper = hyper.hypernyms()[0]\n",
        "\n",
        "traverse(the_chosen_noun)\n",
        "\n",
        "# Read that code aloud and quickly to sound very hyper"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "J9Xo-Rwew9SV",
        "outputId": "b800e7d8-8664-489d-aae5-0d52fd9d5c0a"
      },
      "execution_count": 76,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Synset('canopy.n.03')\n",
            "Synset('shelter.n.02')\n",
            "Synset('protective_covering.n.01')\n",
            "Synset('covering.n.02')\n",
            "Synset('artifact.n.01')\n",
            "Synset('whole.n.02')\n",
            "Synset('object.n.01')\n",
            "Synset('physical_entity.n.01')\n",
            "Synset('entity.n.01')\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "You can see here that there is a clear hiearchy stemming from 'umbrella'. Wordnet's nouns are densly connected in this way, where nouns at the top of the hiearchy are general nouns, with specificity growing as you go down. In that way you could call the noun wordnet a tree. Note that it is also doubly linked, with the ability to traverse in any direction on semantic relations. Those relations are:\n",
        "- hypernyms: A list of synsets that represent more general concepts that the current synset is a type of.\n",
        "- hyponyms: A list of synsets that represent more specific concepts that are types of the current synset.\n",
        "- holonyms: A list of synsets that represent the whole that the current synset is a part of (e.g. 'tree' is a holonym of 'branch').\n",
        "- meronyms: A list of synsets that represent the parts that make up the current synset (e.g. 'branch' is a meronym of 'tree').\n",
        "\n",
        "I could also imagine traversing the net through a words lemmas, attributes, or antonyms.\n",
        "\n",
        "For example, lets look at the relationships to our word!"
      ],
      "metadata": {
        "id": "NaXxwDW-yFgF"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Prints a list attribute of a predefined synset (the_chosen_word) \n",
        "def synset_list(attribute, synset):\n",
        "  if not hasattr(synset, attribute):\n",
        "    print(synset.name() + \" has no attribute: \" + attribute)\n",
        "    return\n",
        "  # Note that I both get the attribute  (a function) and run it with '()'\n",
        "  list = getattr(synset, attribute)()\n",
        "  if not list: \n",
        "    print(\"There are no \" + attribute + \" of the word \" + synset.name())\n",
        "  else:\n",
        "    separator = \", \"\n",
        "    print(attribute + \": \" + \n",
        "          separator.join([element.name() for element in list]))\n",
        "\n",
        "\n",
        "# It's worth noting I was told to ouput an empty list if none exist but this\n",
        "# seems like a better way to communicate to the user if this function were\n",
        "# implemented elsewhere.\n",
        "synset_list(\"hypernyms\", the_chosen_noun)\n",
        "synset_list(\"hyponyms\", the_chosen_noun)\n",
        "synset_list(\"meronyms\", the_chosen_noun)\n",
        "synset_list(\"holonyms\", the_chosen_noun)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "w7Cn6L305FsT",
        "outputId": "df9f223f-3f6b-49d8-f391-4d1bfe68f695"
      },
      "execution_count": 77,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "hypernyms: canopy.n.03\n",
            "hyponyms: gamp.n.01\n",
            "umbrella.n.01 has no attribute: meronyms\n",
            "umbrella.n.01 has no attribute: holonyms\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## The Chosen Verb!\n",
        "Now we move on to verbs! Each part of speech in WordNet has it's own organization, so its worth looking at another section. Asking ChatGPT, our chosen verb is \"Swim\"! I'll go ahead use the lowercase \"swim\". Let's examine it as we did before:"
      ],
      "metadata": {
        "id": "m2xoRMx0CI83"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "the_chosen_verb = 'swim'\n",
        "\n",
        "# All of the synsets\n",
        "print(wn.synsets('swim'))\n",
        "\n",
        "# Just the verbs!\n",
        "synsets = wn.synsets('swim', pos=wn.VERB)\n",
        "print(synsets)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "m1fIj_2qCslD",
        "outputId": "70a31271-0f70-4c92-ce2c-8dd02a0816fd"
      },
      "execution_count": 78,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[Synset('swimming.n.01'), Synset('swim.v.01'), Synset('float.v.02'), Synset('swim.v.03'), Synset('swim.v.04'), Synset('swim.v.05')]\n",
            "[Synset('swim.v.01'), Synset('float.v.02'), Synset('swim.v.03'), Synset('swim.v.04'), Synset('swim.v.05')]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Picking the first synset\n",
        "the_chosen_verb = synsets[0]\n",
        "# The functions from earlier!\n",
        "examine_synset(the_chosen_verb)\n",
        "traverse(the_chosen_verb)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rfa7jPq-DWFe",
        "outputId": "1d6ec42c-0324-4ec7-e417-202b3b377bac"
      },
      "execution_count": 79,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Definition: travel through water\n",
            "Examples: We had to swim for 20 minutes to reach the shore, a big fish was swimming in the tank\n",
            "Lemmas: swim\n",
            "Synset('travel.v.01')\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "wn.synset('travel.v.01').hypernyms()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QlUfMdpEGtda",
        "outputId": "396e7b6c-dc17-4f13-d369-b72f3b833021"
      },
      "execution_count": 80,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[]"
            ]
          },
          "metadata": {},
          "execution_count": 80
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "I couldn't believe that there weren't any more hypernyms for the word swim so I had to check! It goes to show that while wordnet is organized similarly for verbs as it is for nouns, it is less dense so there isn't some *umbrella* noun that generalizes the meaning of every verb. In our case, there isn't a more general action the 'travel'. For an example of a verb hiearchy, think of the word 'whisper', which might have a hiearchy like:\n",
        "\n",
        "`['whisper', 'talk', 'communicate', 'interact']`\n",
        "\n",
        "However, while the tree is less tall, it is more horizontally linked, with more lemmas for a given word. A good example is how each verb could be linked by it's verb_frame, which represents the syntactic pattern a verb might be used in. I'd say noun relationships seem more reliable in my experience, all things considered."
      ],
      "metadata": {
        "id": "cPd7hJDaGWI8"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Morphy?\n",
        "So, usually this tool can find the base form of a word using a set of rules of detachment. In my case, I chose the word 'swim' which is already the base form for the general action of swimming so I can't get any exciting results. I will note, that if I wanted to find every form, I could input the string into morphy, and keep calling morphy with a null string argument, which would keep giving base forms. *Or so I thought*\n",
        "\n",
        "Looking through the NLTK github, it seems the ability to loop through words with multiple base forms is not implemented in the nltk implementation of wordnet! The princeton documentation states:\n",
        "\n",
        "> The first time that Morphy is called with a specific string, it returns a base form. For each subsequent call to Morphy made with a NULL string argument, Morphy returns another base form\n",
        "\n",
        "However, the code of nltk is:\n",
        "\n",
        "```python\n",
        "if pos is None:\n",
        "            morphy = self._morphy\n",
        "            analyses = chain(a for p in POS_LIST for a in morphy(form, p))\n",
        "        else:\n",
        "            analyses = self._morphy(form, pos, check_exceptions)\n",
        "\n",
        "        # get the first one we find\n",
        "        first = list(islice(analyses, 1))\n",
        "        if len(first) == 1:\n",
        "            return first[0]\n",
        "        else:\n",
        "            return None\n",
        "```\n",
        "\n",
        "Which gives no way to access the other analyzed forms! Just the first one returned. However, it reveals we can use a less friendly private function: _morphy. It doesn't add a case for not inputing pos, so we must enter pos. I'll exemplify it's use with axes, which is the example word from the princeton implementation"
      ],
      "metadata": {
        "id": "zCE3WeYqPQS5"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print(wn._morphy('axes', pos='n'))\n",
        "print(wn._morphy('axes', pos='v'))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2d0CEI3YPQ0o",
        "outputId": "6bba8fe4-d90c-4af8-f0d8-663ac0cb7f2f"
      },
      "execution_count": 116,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['ax', 'axis']\n",
            "['axe', 'ax']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "For the word swim, well..."
      ],
      "metadata": {
        "id": "uqnmq0SZYI82"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print(wn._morphy('swim', pos='n'))\n",
        "print(wn._morphy('swim', pos='v'))\n",
        "print(wn._morphy('swim', pos='a'))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WLrJAifxYK30",
        "outputId": "e94fede4-65f0-4ae0-fa91-17c13179e420"
      },
      "execution_count": 117,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['swim']\n",
            "['swim']\n",
            "[]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We find that swim is already a base form (at least in english)."
      ],
      "metadata": {
        "id": "SH6DscH0YhVF"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Similarity and meaning\n",
        "For an example in using wordnet to calculate the meaning of words given their relation to other words, we'll be running the Wu-Palmer similarity metric and Lesk algorithm. For my two words, I think I'll go ahead pick '*weather*' and '*climate*'."
      ],
      "metadata": {
        "id": "QDeI3SPNYlVH"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Checking we have the right synsets\n",
        "weather = wn.synsets('weather', pos='n')[0]\n",
        "print(weather.definition())\n",
        "climate = wn.synsets('climate', pos='n')[0]\n",
        "print(climate.definition())\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6rW82PUZZP6P",
        "outputId": "94ad52a7-2014-4d5d-bf0d-09d91b52d96e"
      },
      "execution_count": 119,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "the atmospheric conditions that comprise the state of the atmosphere in terms of temperature and wind and clouds and precipitation\n",
            "the weather in some location averaged over some long period of time\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Lesk\n",
        "The Lesk algorithm for Word Sense Disambiguation (WSD) takes the context a sentance appears in and gives you the synset with the highest number of overlapping words between the context sentence and the definitions of each synset. Now I know the definition I want to find above, but using lesk lets see if we can find that sysnset with just a sentance.\n",
        "\n",
        " As a human, I know that \"the wonderful weather we are having\" refers to the temperature and etc. But will Lesk see that with just definitions"
      ],
      "metadata": {
        "id": "m1bda0MYa4zQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from nltk.wsd import lesk\n",
        "sent = ['Wonderful', 'weather', 'we', 'are', 'having', '.']\n",
        "print(lesk(sent, 'weather', 'n'))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "B7bHbUekZvpl",
        "outputId": "d1ceadb0-99f8-4565-c5b0-4b414818c099"
      },
      "execution_count": 120,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Synset('weather.n.01')\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "It's right! We were looking for the first definition of weather! What about \"The climate of this place is too hot for me?\" with the word climate"
      ],
      "metadata": {
        "id": "AeVFz5VtcPn7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from nltk import word_tokenize\n",
        "sent = \"The climate of this place is too hot for me?\"\n",
        "token = word_tokenize(sent)\n",
        "print(lesk(token, 'climate', 'n'))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kPO6m09XcaQb",
        "outputId": "0c179cdd-93f1-4daf-fdca-225e2cfd4749"
      },
      "execution_count": 122,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Synset('climate.n.01')\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Right again!\n",
        "\n",
        "Honest I intentionally used context that didn't have too many overlapping words, so the success of the Lesk algorithm was nice and unexpected. Out of curiosity I looked at the code:\n",
        "\n",
        "```python\n",
        "def lesk(context_sentence, ambiguous_word, pos=None, synsets=None):\n",
        "context = set(context_sentence)\n",
        "    if synsets is None:\n",
        "        synsets = wordnet.synsets(ambiguous_word)\n",
        "\n",
        "    if pos:\n",
        "        synsets = [ss for ss in synsets if str(ss.pos()) == pos]\n",
        "\n",
        "    if not synsets:\n",
        "        return None\n",
        "\n",
        "    _, sense = max(\n",
        "        (len(context.intersection(ss.definition().split())), ss) for ss in synsets\n",
        "    )\n",
        "\n",
        "    return sense\n",
        "```\n",
        "\n",
        "And I can see it simply just finds the synset definition with the maximum intersection length... shame! I imagine that since my definitions were the first in the synsets, the algorithm selects them with priority.\n"
      ],
      "metadata": {
        "id": "E01SlpB0c_zF"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Wu Palmer\n",
        "This similarity algorithm compares how similar two word senses are based on the depth of two sens in the taxonomy and that of their least common subsumer (most specific ancestor node). By comparing how much the depth of both differs from their least common 'ancestor' we can see some form of similarity. Words that are more similar, will have LCS closer to both the word's depth. Lets see:\n",
        "\n"
      ],
      "metadata": {
        "id": "GgGHa4ZKfcod"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Fun note from looking at source code, this function is also a \n",
        "# class method of synset (synset.wup_similarity(...) works)\n",
        "print(wn.wup_similarity(weather, climate))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wb2Tu81xgNnY",
        "outputId": "0505887b-8772-429c-f286-6280b925dbe5"
      },
      "execution_count": 123,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0.13333333333333333\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "From that metric, I would have to say I thought they would have to be closer together. If we traverse the tree for each word we can see how distant their common ancestor is:"
      ],
      "metadata": {
        "id": "d8M12rwzhpa8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"Weather: \")\n",
        "traverse(weather)\n",
        "print(\"Climate: \")\n",
        "traverse(climate)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7KCnMO53h-my",
        "outputId": "92a31b8d-b76d-4c6c-c8f0-2d09891688de"
      },
      "execution_count": 125,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Weather: \n",
            "Synset('atmospheric_phenomenon.n.01')\n",
            "Synset('physical_phenomenon.n.01')\n",
            "Synset('natural_phenomenon.n.01')\n",
            "Synset('phenomenon.n.01')\n",
            "Synset('process.n.06')\n",
            "Synset('physical_entity.n.01')\n",
            "Synset('entity.n.01')\n",
            "Climate: \n",
            "Synset('environmental_condition.n.01')\n",
            "Synset('condition.n.01')\n",
            "Synset('state.n.02')\n",
            "Synset('attribute.n.02')\n",
            "Synset('abstraction.n.06')\n",
            "Synset('entity.n.01')\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Wordnet's hypernym relationship doesn't seem to be the best at modeling this. NLTK has more algorithms like Lin Similarity and Jiang-Conrath Similarity that may be better for this kind of relation but it's outside the scope of this experiement."
      ],
      "metadata": {
        "id": "q2wtS10QiXtQ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## SentiWordNet\n",
        "SentiWordNet is a functionality built on top of the word net database that adds a score for emotional sentiment. By knowing if a word is \"positive\", \"negative\", or \"neutral\" you can analyze the sentiment of sentences by the scores of each individual word. This could be applied to analyzing the sentiment of movie reviews or censoring text.\n",
        "\n",
        "For an example, if I pick an emotionally charged word like 'mourn' I could examine it's sentiment using this library."
      ],
      "metadata": {
        "id": "f_XtnemQjvBe"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from nltk.corpus import sentiwordnet as swn\n",
        "nltk.download('sentiwordnet')\n",
        "\n",
        "# I like the second definition word so I am picking index 1\n",
        "print([synset.definition() for synset in wn.synsets(\"mourn\")])\n",
        "mourn = wn.synsets(\"mourn\")[1]\n",
        "print(mourn.name() + \": \" + mourn.definition())\n",
        "\n",
        "sentiment = swn.senti_synset(mourn.name())\n",
        "print(sentiment)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fwmWPtPvkvDN",
        "outputId": "2bad9ba8-e1b8-474b-e471-f202feb783b9"
      },
      "execution_count": 137,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['feel sadness', 'observe the customs of mourning after the death of a loved one']\n",
            "mourn.v.02: observe the customs of mourning after the death of a loved one\n",
            "<mourn.v.02: PosScore=0.0 NegScore=0.0>\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package sentiwordnet to /root/nltk_data...\n",
            "[nltk_data]   Package sentiwordnet is already up-to-date!\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "And look at that... observing the sustoms of mourning after the death of a loved one has no negative connotation... Guess I'll pick the first meaning."
      ],
      "metadata": {
        "id": "afqTN5r4m7ol"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "mourn = wn.synsets(\"mourn\")[0]\n",
        "print(mourn.name() + \": \" + mourn.definition())\n",
        "\n",
        "sentiment = swn.senti_synset(mourn.name())\n",
        "print(sentiment)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zX75zjOAnF3W",
        "outputId": "d20a38dd-f610-4c7f-d3af-6e46ec0ff908"
      },
      "execution_count": 139,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "mourn.v.01: feel sadness\n",
            "<mourn.v.01: PosScore=0.0 NegScore=0.75>\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Overall, the word has a negative polarity. Lets analyze a sentance. I was instructed to output the polarity for each word in the sentence, so  I do. but note that a lot of words just don't have sentiment, and then I also just select the most common (first) sentiment returned by `senti_synsets`."
      ],
      "metadata": {
        "id": "kX3HXcWan-ef"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from tkinter.constants import E\n",
        "sent = \"We return home to mourn the dead\"\n",
        "neg = 0\n",
        "pos = 0\n",
        "tokens = word_tokenize(sent)\n",
        "print(tokens)\n",
        "for token in tokens:\n",
        "  syn_list = list(swn.senti_synsets(token))\n",
        "  if syn_list:\n",
        "    syn = syn_list[0]\n",
        "    print(\"The polarity of\\t\", token, \"is negative:\", syn.neg_score(), \"and positive:\", syn.pos_score())\n",
        "  else:\n",
        "    print(\"No polarity for\\t\", token)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4vNy3axToFqk",
        "outputId": "bb4f9031-7cf4-4990-c18e-752e756bdd08"
      },
      "execution_count": 149,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['We', 'return', 'home', 'to', 'mourn', 'the', 'dead']\n",
            "No polarity for\t We\n",
            "The polarity of\t return is negative: 0.0 and positive: 0.0\n",
            "The polarity of\t home is negative: 0.0 and positive: 0.0\n",
            "No polarity for\t to\n",
            "The polarity of\t mourn is negative: 0.75 and positive: 0.0\n",
            "No polarity for\t the\n",
            "The polarity of\t dead is negative: 0.0 and positive: 0.0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "A complex NLP algorithm would be able to interpret these scores much beyond their basic polarity. I can see an algorithim factoring in sentiment scores into word sense disambiguiation, by choosing a word sense given a found sentiment of the sentence. Or it could at scale be used to categorize the setiment of texts. Large enough corpora could actually reveal complex changes in sentiment over time. Given that sentiment is an aspect of language NLP can struggle with, this kind of encoding has many applications overall."
      ],
      "metadata": {
        "id": "U2PFqAPSqXcE"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Collocations\n",
        "A collocation is when two or more words work together to form some unique meaning, where you can not simply replace one word with a synonym. Words that appear together frequently often take up their own meaning. \"Heavy rain\" is a collocation that could relate to the meaning of the word \"downpour\" and has it's own frequent use in the English langauge.\n",
        "\n",
        "Lets output collocations for text4, the inaugrural corpus within NLTK. We can then select one of the collocations and calculate mutual information."
      ],
      "metadata": {
        "id": "IX0Dc5q1rVnL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Ensure you have nltk.book installed and text4 imported\n",
        "nltk.download(\"book\")\n",
        "from nltk.book import *\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XKGUY8Ncs8cw",
        "outputId": "4f00e2ae-30c2-4090-c7e2-275a3b062622"
      },
      "execution_count": 156,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading collection 'book'\n",
            "[nltk_data]    | \n",
            "[nltk_data]    | Downloading package abc to /root/nltk_data...\n",
            "[nltk_data]    |   Package abc is already up-to-date!\n",
            "[nltk_data]    | Downloading package brown to /root/nltk_data...\n",
            "[nltk_data]    |   Package brown is already up-to-date!\n",
            "[nltk_data]    | Downloading package chat80 to /root/nltk_data...\n",
            "[nltk_data]    |   Package chat80 is already up-to-date!\n",
            "[nltk_data]    | Downloading package cmudict to /root/nltk_data...\n",
            "[nltk_data]    |   Package cmudict is already up-to-date!\n",
            "[nltk_data]    | Downloading package conll2000 to /root/nltk_data...\n",
            "[nltk_data]    |   Package conll2000 is already up-to-date!\n",
            "[nltk_data]    | Downloading package conll2002 to /root/nltk_data...\n",
            "[nltk_data]    |   Package conll2002 is already up-to-date!\n",
            "[nltk_data]    | Downloading package dependency_treebank to\n",
            "[nltk_data]    |     /root/nltk_data...\n",
            "[nltk_data]    |   Package dependency_treebank is already up-to-date!\n",
            "[nltk_data]    | Downloading package genesis to /root/nltk_data...\n",
            "[nltk_data]    |   Package genesis is already up-to-date!\n",
            "[nltk_data]    | Downloading package gutenberg to /root/nltk_data...\n",
            "[nltk_data]    |   Package gutenberg is already up-to-date!\n",
            "[nltk_data]    | Downloading package ieer to /root/nltk_data...\n",
            "[nltk_data]    |   Package ieer is already up-to-date!\n",
            "[nltk_data]    | Downloading package inaugural to /root/nltk_data...\n",
            "[nltk_data]    |   Package inaugural is already up-to-date!\n",
            "[nltk_data]    | Downloading package movie_reviews to\n",
            "[nltk_data]    |     /root/nltk_data...\n",
            "[nltk_data]    |   Package movie_reviews is already up-to-date!\n",
            "[nltk_data]    | Downloading package nps_chat to /root/nltk_data...\n",
            "[nltk_data]    |   Package nps_chat is already up-to-date!\n",
            "[nltk_data]    | Downloading package names to /root/nltk_data...\n",
            "[nltk_data]    |   Package names is already up-to-date!\n",
            "[nltk_data]    | Downloading package ppattach to /root/nltk_data...\n",
            "[nltk_data]    |   Package ppattach is already up-to-date!\n",
            "[nltk_data]    | Downloading package reuters to /root/nltk_data...\n",
            "[nltk_data]    |   Package reuters is already up-to-date!\n",
            "[nltk_data]    | Downloading package senseval to /root/nltk_data...\n",
            "[nltk_data]    |   Package senseval is already up-to-date!\n",
            "[nltk_data]    | Downloading package state_union to /root/nltk_data...\n",
            "[nltk_data]    |   Package state_union is already up-to-date!\n",
            "[nltk_data]    | Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]    |   Package stopwords is already up-to-date!\n",
            "[nltk_data]    | Downloading package swadesh to /root/nltk_data...\n",
            "[nltk_data]    |   Package swadesh is already up-to-date!\n",
            "[nltk_data]    | Downloading package timit to /root/nltk_data...\n",
            "[nltk_data]    |   Package timit is already up-to-date!\n",
            "[nltk_data]    | Downloading package treebank to /root/nltk_data...\n",
            "[nltk_data]    |   Package treebank is already up-to-date!\n",
            "[nltk_data]    | Downloading package toolbox to /root/nltk_data...\n",
            "[nltk_data]    |   Package toolbox is already up-to-date!\n",
            "[nltk_data]    | Downloading package udhr to /root/nltk_data...\n",
            "[nltk_data]    |   Package udhr is already up-to-date!\n",
            "[nltk_data]    | Downloading package udhr2 to /root/nltk_data...\n",
            "[nltk_data]    |   Package udhr2 is already up-to-date!\n",
            "[nltk_data]    | Downloading package unicode_samples to\n",
            "[nltk_data]    |     /root/nltk_data...\n",
            "[nltk_data]    |   Package unicode_samples is already up-to-date!\n",
            "[nltk_data]    | Downloading package webtext to /root/nltk_data...\n",
            "[nltk_data]    |   Package webtext is already up-to-date!\n",
            "[nltk_data]    | Downloading package wordnet to /root/nltk_data...\n",
            "[nltk_data]    |   Package wordnet is already up-to-date!\n",
            "[nltk_data]    | Downloading package wordnet_ic to /root/nltk_data...\n",
            "[nltk_data]    |   Package wordnet_ic is already up-to-date!\n",
            "[nltk_data]    | Downloading package words to /root/nltk_data...\n",
            "[nltk_data]    |   Package words is already up-to-date!\n",
            "[nltk_data]    | Downloading package maxent_treebank_pos_tagger to\n",
            "[nltk_data]    |     /root/nltk_data...\n",
            "[nltk_data]    |   Package maxent_treebank_pos_tagger is already up-\n",
            "[nltk_data]    |       to-date!\n",
            "[nltk_data]    | Downloading package maxent_ne_chunker to\n",
            "[nltk_data]    |     /root/nltk_data...\n",
            "[nltk_data]    |   Package maxent_ne_chunker is already up-to-date!\n",
            "[nltk_data]    | Downloading package universal_tagset to\n",
            "[nltk_data]    |     /root/nltk_data...\n",
            "[nltk_data]    |   Package universal_tagset is already up-to-date!\n",
            "[nltk_data]    | Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]    |   Package punkt is already up-to-date!\n",
            "[nltk_data]    | Downloading package book_grammars to\n",
            "[nltk_data]    |     /root/nltk_data...\n",
            "[nltk_data]    |   Package book_grammars is already up-to-date!\n",
            "[nltk_data]    | Downloading package city_database to\n",
            "[nltk_data]    |     /root/nltk_data...\n",
            "[nltk_data]    |   Package city_database is already up-to-date!\n",
            "[nltk_data]    | Downloading package tagsets to /root/nltk_data...\n",
            "[nltk_data]    |   Package tagsets is already up-to-date!\n",
            "[nltk_data]    | Downloading package panlex_swadesh to\n",
            "[nltk_data]    |     /root/nltk_data...\n",
            "[nltk_data]    |   Package panlex_swadesh is already up-to-date!\n",
            "[nltk_data]    | Downloading package averaged_perceptron_tagger to\n",
            "[nltk_data]    |     /root/nltk_data...\n",
            "[nltk_data]    |   Package averaged_perceptron_tagger is already up-\n",
            "[nltk_data]    |       to-date!\n",
            "[nltk_data]    | \n",
            "[nltk_data]  Done downloading collection book\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "text4.collocation_list()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tKmDKV6euMX6",
        "outputId": "9bf2cb3f-b66d-4c1e-ca9b-4f231c016d94"
      },
      "execution_count": 160,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[('United', 'States'),\n",
              " ('fellow', 'citizens'),\n",
              " ('years', 'ago'),\n",
              " ('four', 'years'),\n",
              " ('Federal', 'Government'),\n",
              " ('General', 'Government'),\n",
              " ('American', 'people'),\n",
              " ('Vice', 'President'),\n",
              " ('God', 'bless'),\n",
              " ('Chief', 'Justice'),\n",
              " ('one', 'another'),\n",
              " ('fellow', 'Americans'),\n",
              " ('Old', 'World'),\n",
              " ('Almighty', 'God'),\n",
              " ('Fellow', 'citizens'),\n",
              " ('Chief', 'Magistrate'),\n",
              " ('every', 'citizen'),\n",
              " ('Indian', 'tribes'),\n",
              " ('public', 'debt'),\n",
              " ('foreign', 'nations')]"
            ]
          },
          "metadata": {},
          "execution_count": 160
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "I feel like picking the \"Almighty God\" collocation for my analysis. Mutual information refers to the point-wise mutual information (pmi). PMI, found via the equation:\n",
        "$$\\log_2\\frac{P(x,y)}{P(x)\\cdot P(y)}$$\n",
        "This translates to the porbability of x and y occurring next to each other, divided by the probability of x multiplied by the probability of y. If the result is positive, then we know that x and y occur more together than expected by chance. And thus we have a collocation. Lets test on 'Almighty' and 'God'"
      ],
      "metadata": {
        "id": "2DgpS_4julWr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from nltk.util import ngrams\n",
        "import math\n",
        "\n",
        "# The number of unigrams. In the text book this number was slightly different\n",
        "# but I don't see why so I am keeping just the length of the text.\n",
        "unigrams = len(text4)\n",
        "\n",
        "# The number of bigrams\n",
        "bigrams = list(ngrams(text4, 2))\n",
        "num_bigrams = len(bigrams)\n",
        "\n",
        "# The probability of x or Almighty\n",
        "px = text4.count(\"Almighty\")/unigrams\n",
        "# print(px)\n",
        "\n",
        "# The probaibility of y or God\n",
        "py = text4.count(\"God\")/unigrams\n",
        "# print(py)\n",
        "\n",
        "# The probability of the bigram of xy or Almighty God\n",
        "pxy = bigrams.count((\"Almighty\", \"God\"))/num_bigrams\n",
        "# print(pxy)\n",
        "\n",
        "# Calculated formula\n",
        "pmi = math.log2(pxy/(px*py))\n",
        "print(\"Our PMI is:\", pmi)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "a1VsiB2nuq3L",
        "outputId": "bb156c08-032d-4a91-9943-cb463ca9bb38"
      },
      "execution_count": 177,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Our PMI is: 9.527367559367379\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The final calculation of the mutual information is rather high, suggesting that (at least in this text) \"Almighty God\" is a collocation. I'd say that this measurement is useful for just finding general sayings in English that aren't very well stated in the grammar and syntax. I know I only ever hear almighty in before God. So much to the point where the name of the film *Bruce Almighty* hints at it's plot involving god with just the name.\n",
        "\n",
        "## Conclusion\n",
        "These are all very good tools. Having a pre-annotated collection of English words that can be used for NLP is very helpful, especially with how easy it is to use the database inside of nltk. I'll have to keep the vocabulary and functions learned in this experiement close to my brain as I implement more and more NLP alorithms."
      ],
      "metadata": {
        "id": "iy75KzAk2ro5"
      }
    }
  ]
}