NLP_Techniques.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyMuQ1zZVuJ/iM4qWtqK8LQG"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# NLP Techniques \n",
        "Based on: https://www.youtube.com/watch?v=M7SWr5xObkA (Keith Galli, PyCon 2020)\n",
        "\n",
        "Additional Resources: \n",
        "* [Natural Language Toolkit - NLTK](https://www.nltk.org/)\n",
        "* [Scikit-learn](https://scikit-learn.org/stable/index.html)\n",
        "    * [scikit-learn text feature extraction](https://scikit-learn.org/stable/modules/feature_extraction.html)\n",
        "    * [The Bag of Words representation](https://scikit-learn.org/stable/modules/feature_extraction.html#the-bag-of-words-representation)\n",
        "    * [CountVector](https://scikit-learn.org/stable/modules/feature_extraction.html#common-vectorizer-usage)\n",
        "    * [scikit-learn SVM](https://scikit-learn.org/https://scikit-learn.org/stable/modules/feature_extraction.html#text-feature-extractionstable/modules/svm.html)\n",
        "* [spaCy](https://spacy.io/)\n",
        "* [Recurrent Neural Networks and Natural Language Processing](https://towardsdatascience.com/recurrent-neural-networks-and-natural-language-processing-73af640c2aa1)\n",
        "* [Regex101](https://regex101.com/)\n",
        "* [Regular Expression Cheat Sheet](https://cheatography.com/davechild/cheat-sheets/regular-expressions/)\n",
        "* [Textblog](https://textblob.readthedocs.io/en/dev/)\n"
      ],
      "metadata": {
        "id": "90gRCyHJBUKL"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Bag of Words Model"
      ],
      "metadata": {
        "id": "ezlguhhh_Uxf"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "In this model, texts are is represented as a bag (multiset) of its words, disregarding grammar and even word order but keeping multiplicity.\n",
        "\n",
        "The bag-of-words model is commonly used in methods of document classification where the (frequency of) occurrence of each word is used as a feature for training a classifier.\n",
        "\n",
        "After transforming the text into a \"bag of words\", we can calculate various measures to characterize the text. The most common type of characteristics, or features calculated from the Bag-of-words model is term frequency, namely, the number of times a term appears in the text.\n",
        "\n",
        "The Bag-of-words model is an orderless document representation — only the counts of words matter.\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "0vsya450tPu7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Define some training utterances (clauses or inputs provided by the user)\n",
        "# Define two classes, one for books and one for clothing\n",
        "\n",
        "class Category:\n",
        "  BOOKS = \"BOOKS\"\n",
        "  CLOTHING = \"CLOTHING\"\n",
        "\n",
        "# for each utterance, assign the appropriate class in order of the list\n",
        "\n",
        "train_x = [\"i love the book\", \"this is a great book\", \"the fit is great\", \"i love the shoes\"]\n",
        "train_y = [Category.BOOKS, Category.BOOKS, Category.CLOTHING, Category.CLOTHING]"
      ],
      "metadata": {
        "id": "lyykoSwgmbCp"
      },
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Vectorization** is the general process of turning a collection of text documents into numerical feature vectors. This specific strategy (tokenization, counting and normalization) is called the Bag of Words or “Bag of n-grams” representation. Documents are described by word occurrences while completely ignoring the relative position information of the words in the document."
      ],
      "metadata": {
        "id": "jwhPEHnJx8LJ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "With Scikitlearn's Text Feature Extraction, we can: \n",
        "\n",
        "* **tokenize** strings and giving an integer id for each p ossible token, for instance by using white-spaces and punctuation as token separators.\n",
        "* **count** the occurrences of tokens in each document.\n",
        "* **normalize** and **weight** with diminishing importance tokens that occur in the majority of samples / documents.\n",
        "\n"
      ],
      "metadata": {
        "id": "XVGfaktLO8Tp"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Vecotorizing will help extract numerical features from text content\n",
        "# CountVectorizer implements both tokenization and occurrence counting in a single class\n",
        "\n",
        "from sklearn.feature_extraction.text import CountVectorizer\n",
        "\n",
        "# Variable for training utterances\n",
        "train_x = [\"i love the book\", \"this is a great book\", \"the fit is great\", \"i love the shoes\"]\n",
        "\n",
        "# Create a Vectorizer Object\n",
        "vectorizer = CountVectorizer()\n",
        "\n",
        "# The fit method is calculating the mean and variance of each of the features present in our data\n",
        "train_x_vectors = vectorizer.fit_transform(train_x)\n",
        "\n",
        "# Get output feature names for transformation.\n",
        "print(vectorizer.get_feature_names_out())\n",
        "\n",
        "# print array for features compared to each utterance\n",
        "print(train_x_vectors.toarray())"
      ],
      "metadata": {
        "id": "o2C2uAmy_ge0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "2b57b44a-0d0e-453b-b8b0-ba82ea7645de"
      },
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['book' 'fit' 'great' 'is' 'love' 'shoes' 'the' 'this']\n",
            "[[1 0 0 0 1 0 1 0]\n",
            " [1 0 1 1 0 0 0 1]\n",
            " [0 1 1 1 0 0 1 0]\n",
            " [0 0 0 0 1 1 1 0]]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "##### Train SVM Model"
      ],
      "metadata": {
        "id": "ql5SlIRlUcPp"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn import svm\n",
        "\n",
        "# SVC (support vector classification)\n",
        "clf_svm = svm.SVC(kernel='linear')\n",
        "\n",
        "    \n",
        "#fit is estimator to be able to predict the classes to which unseen samples belong. \n",
        "# two arguments, our vectors and catagories\n",
        "clf_svm.fit(train_x_vectors, train_y)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dR8NfrDeUbAW",
        "outputId": "2e9961ad-f276-4c53-d2a9-c1a9ae4dce20"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "SVC(kernel='linear')"
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Test new utterance of the trained model"
      ],
      "metadata": {
        "id": "bkyaD_fkWcJB"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "test_x = vectorizer.transform(['i love the book'])\n",
        "\n",
        "clf_svm.predict(test_x)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vG-w9vs8WMyT",
        "outputId": "e9c8d288-5dc5-4f4d-aac0-9ca37d27a2aa"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array(['BOOKS'], dtype='<U8')"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "There are some drawnbacks of the BOW model. For instance, if we change \"book\" to \"story\" in the above test, the prediction is \"clothing.\" "
      ],
      "metadata": {
        "id": "GaCsgSjnWi6e"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# from wiki on BoW and n-gram model\n",
        "\n",
        "from tensorflow import keras\n",
        "from typing import List\n",
        "from keras.preprocessing.text import Tokenizer\n",
        "\n",
        "sentence = [\"John likes to watch movies. Mary likes movies too.\"]\n",
        "\n",
        "def print_bow(sentence: List[str]) -> None:\n",
        "    tokenizer = Tokenizer()\n",
        "    tokenizer.fit_on_texts(sentence)\n",
        "    sequences = tokenizer.texts_to_sequences(sentence)\n",
        "    word_index = tokenizer.word_index \n",
        "    bow = {}\n",
        "    for key in word_index:\n",
        "        bow[key] = sequences[0].count(word_index[key])\n",
        "\n",
        "    print(f\"Bag of word sentence 1:\\n{bow}\")\n",
        "    print(f'We found {len(word_index)} unique tokens.')\n",
        "\n",
        "print_bow(sentence)"
      ],
      "metadata": {
        "id": "L8n88tpYvOg-",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1951f866-28f7-4185-840f-64a8b741dbef"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Bag of word sentence 1:\n",
            "{'likes': 2, 'movies': 2, 'john': 1, 'to': 1, 'watch': 1, 'mary': 1, 'too': 1}\n",
            "We found 7 unique tokens.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Word Vectors\n",
        "Word Embeddings or Word vectorization is a methodology in NLP to map words or phrases from vocabulary to a corresponding vector of real numbers which used to find word predictions, word similarities/semantics.\n",
        "\n",
        "* **continuous bag of word (CBOW Model)**: The distributed representations of context (or surrounding words) are combined to predict the word in the middle. \n",
        "* **skip gram**: The distributed representation of the input word is used to predict the context. \n",
        "\n",
        "Both look at a window of text (e.g., \"Best book I've ever read) and look at different tokens and will utilize the surrounding tokens. For example, \"book\" with \"read\" or \"story\" and \"characters\" might be in a similar in vector space. Then, We can start building out bigger relationships.  \n",
        "</br> \n",
        "[NLP 101: Word2Vec — Skip-gram and CBOW](https://towardsdatascience.com/nlp-101-word2vec-skip-gram-and-cbow-93512ee24314#:~:text=Continuous%20Bag%20of%20Words%20Model%20(CBOW)%20and%20Skip%2Dgram&text=In%20the%20CBOW%20model%2C%20the,used%20to%20predict%20the%20context%20.)"
      ],
      "metadata": {
        "id": "UppJKWKDYGd1"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "spaCy: https://spacy.io/"
      ],
      "metadata": {
        "id": "Ej0qcLlIazz8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        " !pip install Spacy\n",
        " !python -m spacy download en_core_web_md\n",
        "\n"
      ],
      "metadata": {
        "id": "ZG5Qtr-TaNnd"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import spacy \n",
        "nlp = spacy.load(\"en_core_web_md\")"
      ],
      "metadata": {
        "id": "Ugt0tZoNYrMp"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(train_x)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KQbWSfkaXrj7",
        "outputId": "a608dd55-79b3-40c9-dfe2-a4225ad738ab"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['i love the book', 'this is a great book', 'the fit is great', 'i love the shoes']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "docs = [nlp(text) for text in train_x]\n",
        "train_x_word_vectors = [x.vector for x in docs]"
      ],
      "metadata": {
        "id": "vvtOaU4NXvjq"
      },
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn import svm \n",
        "\n",
        "clf_svm_wv = svm.SVC(kernel='linear')\n",
        "clf_svm_wv.fit(train_x_word_vectors, train_y)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZXZR9seTYFnG",
        "outputId": "4759e22f-58e9-4d33-db49-684bbbd8a5b7"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "SVC(kernel='linear')"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# test_x = [\"I love the book\"]\n",
        "# test_x = [\"I love the story\"]\n",
        "test_x = [\"I love the purse\"]\n",
        "test_docs = [nlp(text) for text in test_x]\n",
        "test_x_word_vectors =  [x.vector for x in test_docs]\n",
        "\n",
        "clf_svm_wv.predict(test_x_word_vectors)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dVJ2kk9EawyQ",
        "outputId": "8788d1a0-b123-4d3d-f28a-832cd76fda22"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array(['CLOTHING'], dtype='<U8')"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "A note about word vectors, if we had 10 different catalgories and longer sentences, the meanings might get lost and get less precise. Another drawback is with words with multiple meanings, such as \"check\". (i.e., \"I went to the bank and wrote a check\" and \"let me check that out.\")"
      ],
      "metadata": {
        "id": "-nP6bbZG5HTM"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#Working with Text"
      ],
      "metadata": {
        "id": "D2G2PvqCtyNO"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Regex (Regular Expressions)\n",
        "* Pattern matching of strings \n",
        "* Ccan be used to add, remove, isolate, and manipulate all kinds of text and data.\n",
        "* Regex Cheatsheet: https://cheatography.com/davechild/cheat-sheets/regular-expressions/\n",
        "* Regex101: https://regex101.com/"
      ],
      "metadata": {
        "id": "P8-fA4tNt1oR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# use re.match to find a match in the string\n",
        "# you could also use re.search to check if a regex is anywhere in a string (e.g., when the entirety doesn't need to match)\n",
        "import re\n",
        "\n",
        "regexp = re.compile(r\"^ab[^\\s]*cd$\")\n",
        "phrases = [\"abcd\", \"xxx\", \"abxxcd\", \"ab cd\"]\n",
        "matches = []\n",
        "\n",
        "for phrase in phrases: \n",
        " if re.match(regexp, phrase):\n",
        "  matches.append(phrase)\n",
        "\n",
        "print(matches)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vW0h78oi_Az5",
        "outputId": "2b315ccd-c113-4d88-e758-5713fe7cb159"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['abcd', 'abxxcd']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# connecting regex to the work we've already done \n",
        "\n",
        "# pipe is the \"or\" sign here\n",
        "# \\b is word bountries\n",
        "regexp = re.compile(r\"\\bread\\b|\\bstory\\b|\\bbook\\b\")\n",
        "\n",
        "phrases = [\"I liked that story.\", \"I like that book\", \"this hat is nice\"]\n",
        "\n",
        "matches = []\n",
        "for phrase in phrases:\n",
        "  if re.search(regexp, phrase):\n",
        "    matches.append(phrase)\n",
        "\n",
        "print(matches)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "21qMFn2dCEgn",
        "outputId": "a710066a-8437-4c8d-fdf4-a352cf58f602"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['I liked that story.', 'I like that book']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Stemming and Lemmatization\n",
        "* Stemming: a process that stems or removes last few characters from a word, often leading an incorrect spelling (e.g., \"Caring\" to \"Car\").\t\n",
        "* Lemmatization: considers the context and converts the word to its meaningful base form, which is called Lemma. (e.g., \"Caring\" to \"Care\")\n",
        "\n",
        "Above, especially with BOW, \"book\" and \"books\" are treated differently. However, we can get a \"base\" word to work with. \n",
        "\n",
        "The NLTK Library is helpful for this task."
      ],
      "metadata": {
        "id": "JlNCDD6VtjEf"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import nltk\n",
        "\n",
        "nltk.download('wordnet')\n",
        "nltk.download('stopwords')\n",
        "nltk.download('punkt')\n",
        "nltk.download('omw-1.4')"
      ],
      "metadata": {
        "id": "BjZeJQFBGzuV"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Stemming"
      ],
      "metadata": {
        "id": "wfT3mQM0INMU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from nltk.tokenize import word_tokenize\n",
        "from nltk.stem import PorterStemmer\n",
        "\n",
        "stemmer = PorterStemmer()\n",
        "\n",
        "phrase = \"reading the books\"\n",
        "words = word_tokenize(phrase)\n",
        "\n",
        "stemmed_words = []\n",
        "for word in words:\n",
        "  stemmed_words.append(stemmer.stem(word))\n",
        "\n",
        "\" \".join(stemmed_words)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "id": "fT9mXEWiIMlL",
        "outputId": "1a428893-65e3-4439-e957-79639a839a94"
      },
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'read the book'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Lemmatizing"
      ],
      "metadata": {
        "id": "jrF163_WIWsj"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\n",
        "from nltk.stem import WordNetLemmatizer\n",
        "\n",
        "lemmatizer = WordNetLemmatizer()\n",
        "\n",
        "phrase = \"reading the books\"\n",
        "words = word_tokenize(phrase)\n",
        "\n",
        "lemmatized_words = []\n",
        "for word in words:\n",
        "  lemmatized_words.append(lemmatizer.lemmatize(word, pos='v'))\n",
        "\n",
        "\" \".join(lemmatized_words)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "id": "ApDJDMH8ITBJ",
        "outputId": "aa8294f1-c7c9-4ea2-f88e-d3a06b592639"
      },
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'read the book'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Removing Stop Words \n",
        "#### Tokenize then remove stop word"
      ],
      "metadata": {
        "id": "YmTvKM8iweTp"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from nltk.tokenize import word_tokenize\n",
        "from nltk.corpus import stopwords\n",
        "\n",
        "stop_words = stopwords.words('english')\n",
        "\n",
        "phrase = \"Here is an example sentence demonstrating the removal of stopwords\"\n",
        "\n",
        "words = word_tokenize(phrase)\n",
        "\n",
        "stripped_phrase = []\n",
        "for word in words:\n",
        "  if word not in stop_words:\n",
        "    stripped_phrase.append(word)\n",
        "\n",
        "\" \".join(stripped_phrase)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "id": "TGtOFJzVQpzz",
        "outputId": "d5d8de68-b7d2-41d5-ffe0-259f8825ace3"
      },
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'Here example sentence demonstrating removal stopwords'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Various Other Techniques (Spell Correction, Sentiment, POS Tagging)"
      ],
      "metadata": {
        "id": "KyZu5p9DO530"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!python -m textblob.download_corpora"
      ],
      "metadata": {
        "id": "STWffCaCQ64M"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from textblob import TextBlob\n",
        "\n",
        "phrase = \"the book was horrible\"\n",
        "\n",
        "tb_phrase = TextBlob(phrase)\n",
        "\n",
        "tb_phrase.correct()\n",
        "\n",
        "tb_phrase.tags\n",
        "\n",
        "tb_phrase.sentiment"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IcM9Vb6iQ1b4",
        "outputId": "18e247c0-fe14-4124-f6be-28a88324d305"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Sentiment(polarity=-1.0, subjectivity=1.0)"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Transformer Architecture\n",
        "#### Setup\n"
      ],
      "metadata": {
        "id": "DnI7kMyhRSrG"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install spacy-transformers\n",
        "!python -m spacy download en_core_web_lg"
      ],
      "metadata": {
        "id": "o62_kAJORJrI"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Using Spacy to utilize BERT Model"
      ],
      "metadata": {
        "id": "IlxYnMRsRajP"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import spacy\n",
        "import torch\n",
        "import spacy_transformers\n",
        "\n",
        "nlp = spacy.load(\"en_core_web_lg\")\n",
        "doc = nlp(\"Here is some text to encode.\")"
      ],
      "metadata": {
        "id": "Zyx6X6zsRawg"
      },
      "execution_count": 40,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "class Category:\n",
        "  BOOKS = \"BOOKS\"\n",
        "  BANK = \"BANK\"\n",
        "\n",
        "train_x = [\"good characters and plot progression\", \"check out the book\", \"good story. would recommend\", \"novel recommendation\", \"need to make a deposit to the bank\", \"balance inquiry savings\", \"save money\"]\n",
        "train_y = [Category.BOOKS, Category.BOOKS, Category.BOOKS, Category.BOOKS, Category.BANK, Category.BANK, Category.BANK]\n",
        "     "
      ],
      "metadata": {
        "id": "DlD4PzoRRkVU"
      },
      "execution_count": 41,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn import svm\n",
        "\n",
        "docs = [nlp(text) for text in train_x]\n",
        "train_x_vectors = [doc.vector for doc in docs]\n",
        "clf_svm = svm.SVC(kernel='linear')\n",
        "\n",
        "clf_svm.fit(train_x_vectors, train_y)\n",
        "\n",
        "test_x = [\"check this story out\"]\n",
        "docs = [nlp(text) for text in test_x]\n",
        "test_x_vectors = [doc.vector for doc in docs]\n",
        "\n",
        "clf_svm.predict(test_x_vectors)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "54XzeE_jRng6",
        "outputId": "a98ae2a3-0256-4de2-a5e2-a007070c6fbf"
      },
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array(['BOOKS'], dtype='<U5')"
            ]
          },
          "metadata": {},
          "execution_count": 43
        }
      ]
    }
  ]
}