19. Text Pre-processing spaCy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Text Preprocessing in Python using spaCy library\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "**Here are the topics which we are going to cover:**\n",
    "\n",
    "Tokenization\n",
    "\n",
    "Lemmatization\n",
    "\n",
    "Removing Punctuations and Stopwords\n",
    "\n",
    "Part of Speech Tagging\n",
    "\n",
    "Entity Recognition"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tokenization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Embracing', 'and', 'analyzing', 'self', 'failures', '(', 'of', 'however', 'multitude', ')', 'is', 'a', 'virtue', 'of', 'nobelmen', '.']\n"
     ]
    }
   ],
   "source": [
    "#importing libraries\n",
    "import spacy\n",
    "\n",
    "#instantiating English module\n",
    "nlp = spacy.load(\"en_core_web_sm\")\n",
    "\n",
    "#sample\n",
    "x = \"Embracing and analyzing self failures (of however multitude) is a virtue of nobelmen.\"\n",
    "\n",
    "#creating doc object containing our token features\n",
    "doc = nlp(x)\n",
    "\n",
    "#Creating and updating our list of tokens using list comprehension \n",
    "tokens = [token.text for token in doc]\n",
    "print(tokens)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Below is a sample code for Sentence tokenizing our text.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Embracing and analyzing self failures (of however multitude) is a virtue of nobelmen., And nobility is a treasure few possess.]\n"
     ]
    }
   ],
   "source": [
    "nlp = spacy.load('en_core_web_sm')\n",
    "\n",
    "#Creating the pipeline 'sentencizer' component\n",
    "sbd = nlp.create_pipe('sentencizer')\n",
    "\n",
    "# Adding the component to the pipeline\n",
    "nlp.add_pipe(sbd)\n",
    "\n",
    "text = \"Embracing and analyzing self failures (of however multitude) is a virtue of nobelmen. And nobility is a treasure few possess.\"\n",
    "\n",
    "#creating doc object carring our sentence tokens\n",
    "doc = nlp(text)\n",
    "\n",
    "#Creating and updating our list of tokens using list comprehension \n",
    "tokens = [token for token in doc.sents]\n",
    "print(tokens)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lemmatization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[('Running', 'run'), ('down', 'down'), ('the', 'the'), ('street', 'street'), ('with', 'with'), ('my', '-PRON-'), ('best', 'good'), ('buddy', 'buddy'), ('.', '.')]\n"
     ]
    }
   ],
   "source": [
    "#sample\n",
    "nlp = spacy.load('en_core_web_sm')\n",
    "\n",
    "x = \"Running down the street with my best buddy.\"\n",
    "\n",
    "#creating doc object containing our token features\n",
    "doc = nlp(x)\n",
    "\n",
    "#Creating and updating our list of tokens using list comprehension \n",
    "tokens = [(token.text,token.lemma_) for token in doc]\n",
    "print(tokens)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stop Words"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'whole', 'made', 'herself', 'on', 'neither', 'became', 'see', 'wherever', \"'d\", '’m', 'will', 'whom', 'anyhow', 'even', 'mine', 'five', 'yourselves', 'should', 'than', 'moreover', 'therein', 'around', 'once', 'after', 'both', 'in', 'other', 'whether', 'already', 'three', 'here', 'for', 'eleven', \"'ll\", 'keep', 'anyone', 'rather', 'has', 'at', 'put', 'throughout', 'further', 'just', 'beforehand', 'have', 'somewhere', '‘re', 'get', 'nothing', 'due', 'between', \"'ve\", 'still', 'least', 'indeed', '‘m', 'some', 'may', 'less', '‘ve', 'against', 'us', 'doing', 'is', 'down', 'hers', 'before', 'whoever', 'across', 'either', 'your', 'back', 'none', 'be', 'regarding', 'their', 'them', 'never', 'yours', 'my', 'else', 'also', '’re', 'anywhere', 'n’t', 'nor', 'few', 'although', 'over', 'beside', 'now', 'with', 'it', 'onto', 'she', 'nowhere', '‘ll', 'of', 'much', 'always', 'many', 'former', 'because', 'hereafter', 'about', 'every', 'twelve', 'whose', 'amount', 'out', 'however', 'could', 'therefore', 'side', 'afterwards', 'two', 'sixty', 'something', 'as', 'thereafter', 'six', 'not', 'were', 'very', 'then', 'toward', 'alone', 'sometime', 'anyway', 'yet', 're', 'make', '’d', 'same', 'since', 'herein', 'noone', 'how', 'take', 'myself', 'until', 'say', 'top', 'becoming', 'been', 'latterly', 'unless', 'quite', 'or', 'hence', 'ours', 'besides', 'why', 'bottom', 'cannot', 'call', 'show', 'whereupon', 'really', 'own', 'via', 'too', \"'s\", 'which', 'to', 'formerly', 'elsewhere', 'front', 'him', 'n‘t', 'namely', 'no', 'any', 'meanwhile', 'would', 'thereby', 'amongst', 'whenever', 'am', \"'re\", 'without', 'are', 'each', 'we', 'hundred', 'wherein', 'empty', 'i', 'themselves', 'did', 'they', 'he', 'there', 'beyond', 'under', '’ll', 'thereupon', 'someone', 'whatever', 'hereupon', 'part', 'go', 'and', 'anything', 'what', '‘s', 'others', 'nine', 'whither', \"n't\", 'when', 'everything', 'though', 'these', 'mostly', 'during', 'first', 'please', 'thence', 'his', '‘d', 'her', 'except', 'nobody', 'again', 'sometimes', 'itself', 'if', 'towards', 'along', 'had', 'behind', 'an', 'yourself', 'eight', 'fifteen', 'well', 'thus', 'off', 'thru', 'name', 'up', 'whereafter', 'four', 'more', 'everywhere', 'himself', 'another', 'various', 'from', 'above', 'otherwise', 'using', 'several', 'the', 'most', 'becomes', 'while', 'latter', 'those', 'within', 'its', 'can', 'who', 'through', 'this', 'ca', 'third', 'almost', 'seem', 'often', 'whereby', 'next', 'hereby', 'nevertheless', 'everyone', 'last', 'ourselves', 'fifty', 'ten', 'somehow', 'me', 'whence', 'serious', 'was', 'that', 'where', 'must', 'being', 'done', 'twenty', 'into', 'give', '’ve', 'per', 'so', 'upon', 'seems', 'our', 'below', 'ever', 'might', 'enough', \"'m\", 'among', 'used', 'only', 'a', 'one', 'do', 'together', 'become', 'does', 'such', 'seeming', 'forty', 'by', 'seemed', 'move', 'full', 'all', 'perhaps', 'but', 'whereas', 'you', '’s'}\n"
     ]
    }
   ],
   "source": [
    "from spacy.lang.en.stop_words import STOP_WORDS\n",
    "stop = STOP_WORDS\n",
    "print(stop)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Running', 'down', 'the', 'street', 'with', 'my', 'best', 'buddy', '.']\n",
      "['Running', 'street', 'best', 'buddy', '.']\n"
     ]
    }
   ],
   "source": [
    "#sample\n",
    "x = \"Running down the street with my best buddy.\"\n",
    "\n",
    "#creation of doc object containing our token features\n",
    "\n",
    "doc = nlp(x)\n",
    "\n",
    "#Creating and updating our list of tokens using list comprehension \n",
    "\n",
    "tokens = [token.text for token in doc]\n",
    "\n",
    "print(tokens)\n",
    "\n",
    "#Creating and updating our list of filtered tokens using list comprehension \n",
    "\n",
    "filtered = [token.text for token in doc if token.is_stop == False]\n",
    "\n",
    "print(filtered)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Removing Punctuation\n",
    "\n",
    "You can compare the above two lists and notice words such as down,the,with and my have been removed.Now, similarly, we can also remove punctuation from our text as well using \"isalpha\" method of string objects and using list comprehensions\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['BLIMEY', '!', '!', 'Such', 'an', 'exhausting', 'day', ',', 'I', 'ca', \"n't\", 'even', 'describe', '.']\n",
      "['BLIMEY', 'exhausting', 'day', 'describe'] \n",
      "\n",
      "BLIMEY exhausting day describe\n"
     ]
    }
   ],
   "source": [
    "#sample \n",
    "x = \"BLIMEY!! Such an exhausting day, I can't even describe.\"\n",
    "\n",
    "#creation of doc object containing our token features\n",
    "doc = nlp(x)\n",
    "\n",
    "#Unfiltered tokens \n",
    "tokens = [token.text for token in doc]\n",
    "print(tokens)\n",
    "\n",
    "#Filtering our tokens\n",
    "filtered = [token.text for token in doc if token.is_stop == False and       \n",
    "token.text.isalpha() == True]\n",
    "\n",
    "print(filtered,\"\\n\")\n",
    "print(\" \".join(filtered))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part-of-Speech Tagging (POS)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[('Robin', 'PROPN'), ('is', 'AUX'), ('an', 'DET'), ('astute', 'NOUN'), ('programmer', 'NOUN')]\n"
     ]
    }
   ],
   "source": [
    "#sample\n",
    "x = \"Robin is an astute programmer\"\n",
    "\n",
    "#Creating doc object\n",
    "doc = nlp(x)\n",
    "\n",
    "#Extracting POS\n",
    "pos = [(token.text,token.pos_) for token in doc]\n",
    "print (pos)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Entity Recognition\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(India, 'GPE', 384), (as much as 3, 'CARDINAL', 397), ($39 billion, 'MONEY', 394), (Asia, 'LOC', 385), (third, 'ORDINAL', 396)]\n"
     ]
    }
   ],
   "source": [
    "#sample article\n",
    "x = u\"\"\" India is considering a proposal to guarantee as much as 3  \n",
    "trillion rupees ($39 billion) of loans to small businesses as part of a plan to \n",
    "restart Asia's third-largest economy, which is reeling under the impact of a 40-\n",
    "day lockdown, people with knowledge of the matter said.\"\"\"\n",
    "\n",
    "#creating doc object\n",
    "bloomberg= nlp(x)\n",
    "\n",
    "\n",
    "#extracting entities \n",
    "entities=[(i, i.label_, i.label) for i in bloomberg.ents]\n",
    "print(entities)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## testing all these techniques on our email-spam detection data set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "df=pd.read_csv(\"/Users/sudhanshukumar/Documents/Development/Machine Learning/0 csv files/SMSSpamCollection.csv\",sep=\"\\t\",names=[\"label\",\"message\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>message</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ham</td>\n",
       "      <td>Go until jurong point, crazy.. Available only ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ham</td>\n",
       "      <td>Ok lar... Joking wif u oni...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>spam</td>\n",
       "      <td>Free entry in 2 a wkly comp to win FA Cup fina...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ham</td>\n",
       "      <td>U dun say so early hor... U c already then say...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ham</td>\n",
       "      <td>Nah I don't think he goes to usf, he lives aro...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  label                                            message\n",
       "0   ham  Go until jurong point, crazy.. Available only ...\n",
       "1   ham                      Ok lar... Joking wif u oni...\n",
       "2  spam  Free entry in 2 a wkly comp to win FA Cup fina...\n",
       "3   ham  U dun say so early hor... U c already then say...\n",
       "4   ham  Nah I don't think he goes to usf, he lives aro..."
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "def text_process(message):\n",
    "    doc=nlp(message)\n",
    "    final=[token.lemma_ for token in doc if token.is_stop== False and token.text.isalpha()== True ]\n",
    "    return final\n",
    "#     return \" \".join(final)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['embrace', 'analyze', 'self', 'failure', 'multitude', 'virtue', 'nobelman', 'nobility', 'treasure', 'possess'] \n",
      "\n",
      "embrace analyze self failure multitude virtue nobelman nobility treasure possess\n"
     ]
    }
   ],
   "source": [
    "text = \"Embracing AND analyzing self failures (of however multitude) is a virtue of nobelmen. And nobility is a treasure few possess.\"\n",
    "print(text_process(text),\"\\n\")\n",
    "print(\" \".join(text_process(text)))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    [jurong, point, crazy, available, bugis, n, gr...\n",
       "1                       [ok, lar, joking, wif, u, oni]\n",
       "2    [free, entry, wkly, comp, win, FA, Cup, final,...\n",
       "Name: message, dtype: object"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"message\"].head(3).apply(text_process)  #apply func works on DataFrames"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Detect & remove empty strings\n",
    "Technically, we're dealing with \"whitespace only\" strings. If the original .tsv file had contained empty strings, pandas **.read_csv()** would have assigned NaN values to those cells by default.\n",
    "\n",
    "In order to detect these strings we need to iterate over each row in the DataFrame. The **.itertuples()** pandas method is a good tool for this as it provides access to every field. For brevity we'll assign the names `i`, `lb` and `rv` to the `index`, `label` and `review` columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 blanks:  []\n"
     ]
    }
   ],
   "source": [
    "blanks = []  # start with an empty list\n",
    "\n",
    "for i,lb,msg in df.itertuples():  # iterate over the DataFrame\n",
    "    if type(msg)==str:            # avoid NaN values\n",
    "        if msg.isspace():         # test 'review' for whitespace\n",
    "            blanks.append(i)     # add matching index numbers to the list\n",
    "        \n",
    "print(len(blanks), 'blanks: ', blanks)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from sklearn.svm import LinearSVC\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "tfidf=TfidfVectorizer(analyzer=text_process)\n",
    "\n",
    "X=tfidf.fit_transform(df[\"message\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "y=df[\"label\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9802690582959641"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf=LinearSVC()\n",
    "\n",
    "clf.fit(X_train,y_train)\n",
    "clf.score(X_test,y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9838565022421525"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#without analyzer\n",
    "\n",
    "tfidf=TfidfVectorizer()\n",
    "\n",
    "X=tfidf.fit_transform(df[\"message\"])\n",
    "y=df[\"label\"]\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
    "\n",
    "clf=LinearSVC()\n",
    "\n",
    "clf.fit(X_train,y_train)\n",
    "clf.score(X_test,y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9901345291479821"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cv=CountVectorizer()\n",
    "\n",
    "X=cv.fit_transform(df[\"message\"])\n",
    "y=df[\"label\"]\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
    "\n",
    "clf=LinearSVC()\n",
    "\n",
    "clf.fit(X_train,y_train)\n",
    "clf.score(X_test,y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"processed\"]=df[\"message\"].apply(text_process)  #uncomment the last line of text_process func"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>message</th>\n",
       "      <th>processed</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ham</td>\n",
       "      <td>Go until jurong point, crazy.. Available only ...</td>\n",
       "      <td>jurong point crazy available bugis n great wor...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ham</td>\n",
       "      <td>Ok lar... Joking wif u oni...</td>\n",
       "      <td>ok lar joking wif u oni</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>spam</td>\n",
       "      <td>Free entry in 2 a wkly comp to win FA Cup fina...</td>\n",
       "      <td>free entry wkly comp win FA Cup final tkts Tex...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ham</td>\n",
       "      <td>U dun say so early hor... U c already then say...</td>\n",
       "      <td>U dun early hor U c</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ham</td>\n",
       "      <td>Nah I don't think he goes to usf, he lives aro...</td>\n",
       "      <td>nah think go usf live</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  label                                            message  \\\n",
       "0   ham  Go until jurong point, crazy.. Available only ...   \n",
       "1   ham                      Ok lar... Joking wif u oni...   \n",
       "2  spam  Free entry in 2 a wkly comp to win FA Cup fina...   \n",
       "3   ham  U dun say so early hor... U c already then say...   \n",
       "4   ham  Nah I don't think he goes to usf, he lives aro...   \n",
       "\n",
       "                                           processed  \n",
       "0  jurong point crazy available bugis n great wor...  \n",
       "1                            ok lar joking wif u oni  \n",
       "2  free entry wkly comp win FA Cup final tkts Tex...  \n",
       "3                                U dun early hor U c  \n",
       "4                              nah think go usf live  "
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}