week_1/building-the-vocabulary.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# NLP Course 2 Week 1 Lesson : Building The Model - Lecture Exercise 01\n",
    "Estimated Time: 10 minutes\n",
    "<br>\n",
    "# Vocabulary Creation \n",
    "Create a tiny vocabulary from a tiny corpus\n",
    "<br>\n",
    "It's time to start small !\n",
    "<br>\n",
    "### Imports and Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# imports\n",
    "import re # regular expression library; for tokenization of words\n",
    "from collections import Counter # collections library; counter: dict subclass for counting hashable objects\n",
    "import matplotlib.pyplot as plt # for data visualization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "red pink pink blue blue yellow ORANGE BLUE BLUE PINK black black\n",
      "string length : 64\n"
     ]
    }
   ],
   "source": [
    "# the tiny corpus of text ! \n",
    "text = 'red pink pink blue blue yellow ORANGE BLUE BLUE PINK black black' # 🌈\n",
    "print(text)\n",
    "print('string length :', len(text))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preprocessing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "red pink pink blue blue yellow orange blue blue pink black black\n",
      "string length : 64\n"
     ]
    }
   ],
   "source": [
    "# convert all letters to lower case\n",
    "text_lowercase = text.lower()\n",
    "print(text_lowercase)\n",
    "print('string length :', len(text_lowercase))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['red', 'pink', 'pink', 'blue', 'blue', 'yellow', 'orange', 'blue', 'blue', 'pink', 'black', 'black']\n",
      "count : 12\n"
     ]
    }
   ],
   "source": [
    "# some regex to tokenize the string to words and return them in a list\n",
    "words = re.findall(r'\\w+', text_lowercase)\n",
    "print(words)\n",
    "print('count :', len(words))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create Vocabulary\n",
    "Option 1 : A set of distinct words from the text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'yellow', 'pink', 'red', 'orange', 'blue'}\n",
      "count : 5\n"
     ]
    }
   ],
   "source": [
    "# create vocab\n",
    "vocab = set(words)\n",
    "print(vocab)\n",
    "print('count :', len(vocab))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Add Information with Word Counts\n",
    "Option 2 : Two alternatives for including the word count as well"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'red': 1, 'pink': 3, 'blue': 4, 'yellow': 1, 'orange': 1}\n",
      "count : 5\n"
     ]
    }
   ],
   "source": [
    "# create vocab including word count\n",
    "counts_a = dict()\n",
    "for w in words:\n",
    "    counts_a[w] = counts_a.get(w, 0) + 1\n",
    "print(counts_a)\n",
    "print('count :', len(counts_a))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counter({'blue': 4, 'pink': 3, 'black': 2, 'red': 1, 'yellow': 1, 'orange': 1})\n",
      "count :  6\n"
     ]
    }
   ],
   "source": [
    "# create vocab including word count using collections.Counter\n",
    "counts_b = Counter(words)\n",
    "print(counts_b)\n",
    "print('count : ',len(counts_b))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAATLklEQVR4nO3dfbBc9X3f8ffHslJojId6dBOoHpDbEjfGNQ++yDA4DXFtDxBa6pam0CTEdBoNLh7jadLWSRpcJpNpp9NJW5CDotoUmDw4Tux4NETEprUpkEYYSRayQTDVuGbQoDEydoRlGFPot3/skbNedu+eK+3VRT/er5mdex5+e/b7u7v3c8/93fOQqkKSdOJ7zXIXIEmaDQNdkhphoEtSIwx0SWqEgS5JjXjtcr3wqlWrav369cv18pJ0Qtq5c+c3qmpu3LplC/T169ezY8eO5Xp5STohJXli0jqHXCSpEQa6JDXCQJekRhjoktQIA12SGmGgS1Ijegd6khVJvpTkrjHrkuTmJPuS7Ely3mzLlCRNs5g99BuAvRPWXQqc2T02ArceY12SpEXqFehJ1gA/CXxsQpMrgDtrYDtwapLTZ1SjJKmHvmeK/mfgXwGnTFi/GnhyaH5/t+zAcKMkGxnswbNu3brF1Pl9kqN+6nHlvUMkHU9T99CTXA48XVU7F2o2ZtnL4qyqtlTVfFXNz82NvRSBJOko9RlyuQj4e0m+BnwCeGeS3x5psx9YOzS/BnhqJhVKknqZGuhV9UtVtaaq1gNXAZ+vqp8ZabYVuKY72uUC4FBVHRjdliRp6Rz11RaTXAdQVZuBbcBlwD7gOeDamVQnSeptUYFeVfcC93bTm4eWF3D9LAuTJC2OZ4pKUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhrR5ybRJyX5YpKHkzyS5KYxbS5OcijJ7u5x49KUK0mapM8di74LvLOqDidZCTyQ5O6q2j7S7v6qunz2JUqS+pga6N3t5Q53syu7Ry1lUZKkxes1hp5kRZLdwNPAPVX14JhmF3bDMncnOWuWRUqSpusV6FX1UlWdA6wBNiR5y0iTXcAZVXU2cAvwmXHbSbIxyY4kOw4ePHj0VUuSXmZRR7lU1Z8D9wKXjCx/tqoOd9PbgJVJVo15/paqmq+q+bm5uaMuWpL0cn2OcplLcmo3fTLwLuCxkTanJUk3vaHb7jMzr1aSNFGfo1xOB+5IsoJBUH+yqu5Kch1AVW0GrgTen+RF4Hngqu6fqZKk46TPUS57gHPHLN88NL0J2DTb0iRJi+GZopLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktSIPvcUPSnJF5M8nOSRJDeNaZMkNyfZl2RPkvOWplxJ0iR97in6XeCdVXU4yUrggSR3V9X2oTaXAmd2j7cDt3ZfJUnHydQ99Bo43M2u7B6jN4C+Ariza7sdODXJ6bMtVZK0kD576CRZAewE/gbw0ap6cKTJauDJofn93bIDI9vZCGwEWLdu3VGW3Kj/uWO5K+jnx+eXuwJJE/T6p2hVvVRV5wBrgA1J3jLSJOOeNmY7W6pqvqrm5+bmFl2sJGmyRR3lUlV/DtwLXDKyaj+wdmh+DfDUsRQmSVqcPke5zCU5tZs+GXgX8NhIs63ANd3RLhcAh6rqAJKk46bPGPrpwB3dOPprgE9W1V1JrgOoqs3ANuAyYB/wHHDtEtUrSZpgaqBX1R7g3DHLNw9NF3D9bEuTJC2GZ4pKUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSI/rcU3Rtki8k2ZvkkSQ3jGlzcZJDSXZ3jxuXplxJ0iR97in6IvALVbUrySnAziT3VNWjI+3ur6rLZ1+iJKmPqXvoVXWgqnZ1098G9gKrl7owSdLiLGoMPcl6BjeMfnDM6guTPJzk7iRnTXj+xiQ7kuw4ePDg4quVJE3UO9CTvA74FPChqnp2ZPUu4IyqOhu4BfjMuG1U1Zaqmq+q+bm5uaMsWZI0Tq9AT7KSQZj/TlV9enR9VT1bVYe76W3AyiSrZlqpJGlBfY5yCfBxYG9V/caENqd17UiyodvuM7MsVJK0sD5HuVwE/Czw5SS7u2W/DKwDqKrNwJXA+5O8CDwPXFVVNftyJUmTTA30qnoAyJQ2m4BNsypKkrR4nikqSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjehzT9G1Sb6QZG+SR5LcMKZNktycZF+SPUnOW5pyJUmT9Lmn6IvAL1TVriSnADuT3FNVjw61uRQ4s3u8Hbi1+ypJOk6m7qFX1YGq2tVNfxvYC6weaXYFcGcNbAdOTXL6zKuVJE3UZw/9e5KsB84FHhxZtRp4cmh+f7fswMjzNwIbAdatW7fIUnUiSRa8r/grRlUtdwnSzPT+p2iS1wGfAj5UVc+Orh7zlJf9pFTVlqqar6r5ubm5xVUqSVpQr0BPspJBmP9OVX16TJP9wNqh+TXAU8deniSprz5HuQT4OLC3qn5jQrOtwDXd0S4XAIeq6sCEtpKkJdBnDP0i4GeBLyfZ3S37ZWAdQFVtBrYBlwH7gOeAa2deqSRpQVMDvaoeYPwY+XCbAq6fVVGSpMXzTFFJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqRJ97it6W5OkkX5mw/uIkh5Ls7h43zr5MSdI0fe4pejuwCbhzgTb3V9XlM6lIknRUpu6hV9V9wDePQy2SpGMwqzH0C5M8nOTuJGdNapRkY5IdSXYcPHhwRi8tSYLZBPou4IyqOhu4BfjMpIZVtaWq5qtqfm5ubgYvLUk64pgDvaqerarD3fQ2YGWSVcdcmSRpUY450JOcliTd9IZum88c63YlSYsz9SiXJL8HXAysSrIf+AiwEqCqNgNXAu9P8iLwPHBVVdWSVSxJGmtqoFfV1VPWb2JwWKMkaRl5pqgkNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1YmqgJ7ktydNJvjJhfZLcnGRfkj1Jzpt9mZKkafrsod8OXLLA+kuBM7vHRuDWYy9LkrRYUwO9qu4DvrlAkyuAO2tgO3BqktNnVaAkqZ+pN4nuYTXw5ND8/m7ZgdGGSTYy2Itn3bp1M3hp6ThKlruCfqp6NjxB+kPf/gC/e4L06Z8sok+LMIt/io77Do6ttqq2VNV8Vc3Pzc3N4KUlSUfMItD3A2uH5tcAT81gu5KkRZhFoG8FrumOdrkAOFRVLxtukSQtralj6El+D7gYWJVkP/ARYCVAVW0GtgGXAfuA54Brl6pYSdJkUwO9qq6esr6A62dWkSTpqHimqCQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDWiV6AnuSTJ40n2JfnwmPUXJzmUZHf3uHH2pUqSFtLnnqIrgI8C7wb2Aw8l2VpVj440vb+qLl+CGiVJPfTZQ98A7Kuqr1bVC8AngCuWtixJ0mL1CfTVwJND8/u7ZaMuTPJwkruTnDVuQ0k2JtmRZMfBgwePolxJ0iR9Aj1jltXI/C7gjKo6G7gF+My4DVXVlqqar6r5ubm5RRUqSVpYn0DfD6wdml8DPDXcoKqerarD3fQ2YGWSVTOrUpI0VZ9Afwg4M8kbk/wAcBWwdbhBktOSpJve0G33mVkXK0mabOpRLlX1YpIPAJ8FVgC3VdUjSa7r1m8GrgTen+RF4HngqqoaHZaRJC2hqYEO3xtG2TaybPPQ9CZg02xLkyQthmeKSlIjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiN6BXqSS5I8nmRfkg+PWZ8kN3fr9yQ5b/alSpIWMjXQk6wAPgpcCrwZuDrJm0eaXQqc2T02ArfOuE5J0hR99tA3APuq6qtV9QLwCeCKkTZXAHfWwHbg1CSnz7hWSdIC+twkejXw5ND8fuDtPdqsBg4MN0qykcEePMDhJI8vqtqltQr4xiw3mMxya0dl5n1aZkvwHi37mzT792h5+7QEn7kG36OfPqY+nTFpRZ9AH/fKdRRtqKotwJYer3ncJdlRVfPLXccstdan1voD7fWptf7AidWnPkMu+4G1Q/NrgKeOoo0kaQn1CfSHgDOTvDHJDwBXAVtH2mwFrumOdrkAOFRVB0Y3JElaOlOHXKrqxSQfAD4LrABuq6pHklzXrd8MbAMuA/YBzwHXLl3JS+YVORR0jFrrU2v9gfb61Fp/4ATqU6peNtQtSToBeaaoJDXCQJekRjQf6EnWJ/nKmOX3JjkhDkXqK8nHxpzFO9rm9iRXHq+aFjLr9ybJ+5Jsmk11yyvJv03yi8tdRx9JDndfx76fOn76HIeuE0RV/bPlrkELy+BMplTV/1vuWtTe+9H8HnrntUnu6C4c9odJ/vLwyiN7GN30lUlu76bnknwqyUPd46LjXPdY3Z7QY6N9Gt6zTXI4ya8neTjJ9iQ/PGY7v9btsS/n52Dae3Nrkh1JHkly09Dy85P8r65/X0xyysjzfjLJnyVZdbw6Mkn3fu1N8pvALuBXu8/TnpE+/Up3Ebz/DrxpGev9tSQ3DM3/epIPJvmX4+qesI2Tkvy3JF9O8qUkP9Et35bkrd30l5LcOPSaS7JDkuRfJPlK9/jQmPdj7QKfs68luSnJrq4vf7NbPpfknm75byV54shnLcnPdJ/J3d26FUvRr3FeLYH+JmBLVb0VeBb45z2f91+A/1RV5wP/EPjYEtV3NKb16QeB7VV1NnAf8PPDK5P8B+CHgGuXee9kWj9+pTtL763Ajyd5awbnQ/w+cEPXv3cBzx95QpL3Ah8GLquqV8qlD94E3An8awaXxdgAnAO8LcnfTvI2Bud4nAv8A+D8ZaoT4OPAzwF0v+yvAr7O4OJ731f3Atu4HqCq/hZwNXBHkpMYfBZ/LMnrgReBIztJ7wDun3VHuu/rtQwuV3IBg5+Dv0L3flTVuVX1BGM+Z0Ob+UZVncfgooNHhsE+Any+W/5HwLru9X4U+MfARVV1DvAS8NOz7tckr5Yhlyer6k+76d8GPtjzee8C3py/uDbG65OcUlXfnnWBR2Fan14A7uqmdwLvHlr3q8CDVbWR5TetHz+VwTWAXguczuCKnwUcqKqHAKrqWfjedVl+ApgH3nNk+SvEE1W1Pcl/BN4DfKlb/joGQXkK8EdV9RxAktGT946bqvpakmeSnAv8MINaz2d83fdN2Mw7gFu67T2W5AngRxiE9geB/wP8MfDu7q+y9VW1FNd2egeD7+t3AJJ8GvgxuvdjqN24z9mebt2nu687GfyyPbLd93b9+5Mk3+qW/x3gbcBD3efxZODpJejXWK+WQB892H6h+ZOGpl8DXFhVz/PKM61P/7f+4iSDl/j+9/ohBntYb6iqby5VgT1N7EeSNzLYIzq/qr6VwVDYSQyuHTTpBIqvAn+NQXjsmHm1R+873dcA/66qfmt4ZZIPMblPy+FjwPuA04DbGATVy+pewKSrTz3E4BfuV4F7GFz46ucZhOVSmFTHd77XYPLn7Ijvdl+Hf44mbTfAHVX1S0dd8TF4tQy5rEtyYTd9NfDAyPqvJ/nR7s/L9w4t/xzwgSMzSc5Z0ioXZ1qfFvInwL8H/nh07HkZLNSP1zP4wTvU/Q/g0m75Y8BfTXI+QJJTkhz5QXuCwV7UnUnOWvLqF++zwD9N8jqAJKuT/BCDPd33Jjm5e0/+7nIWyWAY4RIGe+afZXLdk9xHN9SQ5EcYDEk83l2C+0ngp4DtDPbYf5ElGG4ZquPvZ/A/ph9k8PM9+lqTPmcLeYBBH0jyHgbDOAD/A7jyyPcmyRuSTLw64qy9WgJ9L/BzSfYAb+DlN+D4MIPhic/z/Zf8/SAw3/0T6FHguuNRbE/T+rSgqvoD4L8CW5OcvAT19TWxH1X1MIM/8R9hsJf4p93yFxiMU96S5GEGe3onDT3vcQZh8gdJ/vpx6kcvVfU54HeBP0vyZeAPgVOqaheD/wvsBj7F0gVcL933+AvAJ6vqpUl1L7CJ3wRWdG1/H3hfVR3Z070f+Ho3vHQ/g4v5LUl/u+/r7cAXgQcZ/OXxrZE2Yz9nU9wEvCfJLga/AA4A366qR4F/A3yu+0zfw2AI57jw1P8TUJL1wF1V9ZblrkVt6v5a3QX8o6r638tdzytNkr8EvNRd6+pC4Nbun6DL6tUyhi6ppwxOTruLwT8TDfPx1gGf7H7xvcDIUWTLxT10SWrEq2UMXZKaZ6BLUiMMdElqhIEuSY0w0CWpEf8f0F9c7+r+NGMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# barchart of sorted word counts\n",
    "d = {\n",
    "    'blue': counts_b['blue'], \n",
    "     'pink': counts_b['pink'], \n",
    "     'black': counts_b['black'],\n",
    "     'red': counts_b['red'], \n",
    "     'yellow': counts_b['yellow'], \n",
    "    'orange': counts_b['orange']\n",
    "}\n",
    "plt.bar(range(len(d)), list(d.values()), align='center', color=d.keys())\n",
    "_ = plt.xticks(range(len(d)), list(d.keys()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Ungraded Exercise\n",
    "Note that `counts_b`, above, returned by `collections.Counter` is sorted by word count\n",
    "\n",
    "Can you modify the tiny corpus of ***text*** so that a new color appears \n",
    "between ***pink*** and ***red*** in `counts_b` ?\n",
    "\n",
    "Do you need to run all the cells again, or just specific ones ? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "counts_b :  Counter({'blue': 4, 'pink': 3, 'black': 2, 'red': 1, 'yellow': 1, 'orange': 1})\n",
      "count :  6\n"
     ]
    }
   ],
   "source": [
    "print('counts_b : ', counts_b)\n",
    "print('count : ', len(counts_b))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected Outcome:\n",
    "\n",
    "counts_b : Counter({'blue': 4, 'pink': 3, **'your_new_color_here': 2**, red': 1, 'yellow': 1, 'orange': 1})\n",
    "<br>\n",
    "count :  6"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Summary\n",
    "\n",
    "This is a tiny example but the methodology scales very well.\n",
    "<br>\n",
    "In the assignment you will create a large vocabulary of thousands of words, from a corpus\n",
    "<br>\n",
    "of tens of thousands or words! But the mechanics are exactly the same. \n",
    "<br> \n",
    "The only extra things to pay attention to should be; run time, memory management and the vocab data structure.\n",
    "<br> \n",
    "So the choice of approach used in code blocks `counts_a` vs `counts_b`, above, will be important."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}