ml-firstnames-classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Machine Learning: Classifying First Names by Gender\n",
    "\n",
    "## Goal\n",
    "\n",
    "We will try to predict whether a first name is a typical boy's or girl's name, using machine learning.\n",
    "\n",
    "This workshop is a bit related to the demo James gave last week. In his demo he made the computer recognize whether he was holding an apple, a banana, or some other fruit. All done with just the click of a couple of buttons!\n",
    "\n",
    "But a computer can't really see or understand what it sees like we humans do. In the end the thing a computer only really does is compute. It needs numbers! So how does the image of James holding a banana become just numbers a computer can \"understand\"?\n",
    "\n",
    "Well, that's [quite complicated](https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_feature2d/py_table_of_contents_feature2d/py_table_of_contents_feature2d.html) involving some complex math. So today we start with text rather than images, which is a bit easier and good way to introduce _feature extraction_. Start simple, right?\n",
    "\n",
    "So our goals for today are:\n",
    "1. Make the computer \"understand\" names by expressing names as numbers\n",
    "2. Let the computer figure out what's a typical boys or girls name.\n",
    "3. Ask the computer whether a name it has never seen is a tyical boys or girls name.\n",
    "\n",
    "## Before we begin\n",
    "Let's make this a bit more interactive. You can follow along and play with the code online, or with your own files.\n",
    "\n",
    "### Play Online\n",
    "You can follow along and play with the code here:  \n",
    "https://mybinder.org/v2/gh/jelmervdl/prewired/master?filepath=machine-learning.ipynb\n",
    "\n",
    "### Play with your own files\n",
    "\n",
    "We need some data to play with:\n",
    "1. https://github.com/teropa/nlp/raw/master/resources/corpora/names/female.txt\n",
    "2. https://github.com/teropa/nlp/raw/master/resources/corpora/names/male.txt\n",
    "\n",
    "Download these files and put them in a place you can easily access from Python. You will also need matplotlib and sklearn if you want to run the code here. You can run this on the command line:\n",
    "```\n",
    "pip3 install scikit-learn matplotlib\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data exploration\n",
    "Let's see what we've got here. Machine learning, or any data science, always begins with understanding what you are working with."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's read the names into Python. This code will open the file, read each of the names from the file, and remove any spaces or line breaks from the name so we just have the name, nothing more."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(\"male.txt\") as file:\n",
    "    male_names = [name.strip() for name in file]\n",
    "\n",
    "with open(\"female.txt\") as file:\n",
    "    female_names = [name.strip() for name in file]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_How many? Many!_ Let's check how much data we have."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 191,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2943, 5001)"
      ]
     },
     "execution_count": 191,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(male_names), len(female_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Apparently many more female names than male names. Interesting…"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start by printing some of those names, for example the first ten."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 192,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(['Aamir',\n",
       "  'Aaron',\n",
       "  'Abbey',\n",
       "  'Abbie',\n",
       "  'Abbot',\n",
       "  'Abbott',\n",
       "  'Abby',\n",
       "  'Abdel',\n",
       "  'Abdul',\n",
       "  'Abdulkarim'],\n",
       " ['Abagael',\n",
       "  'Abagail',\n",
       "  'Abbe',\n",
       "  'Abbey',\n",
       "  'Abbi',\n",
       "  'Abbie',\n",
       "  'Abby',\n",
       "  'Abigael',\n",
       "  'Abigail',\n",
       "  'Abigale'])"
      ]
     },
     "execution_count": 192,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "male_names[:10], female_names[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The names are sorted alphabetically. To make things a bit more interesting, shuffle them around *but don't mix them*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 193,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "random.shuffle(male_names)\n",
    "random.shuffle(female_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 194,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(['Ravi',\n",
       "  'Chrisy',\n",
       "  'Jackie',\n",
       "  'Harrison',\n",
       "  'Talbot',\n",
       "  'Kermit',\n",
       "  'Munmro',\n",
       "  'Avery',\n",
       "  'Merry',\n",
       "  'Reinhold'],\n",
       " ['Carlita',\n",
       "  'Erna',\n",
       "  'Sherill',\n",
       "  'Oprah',\n",
       "  'Katee',\n",
       "  'Muire',\n",
       "  'Minta',\n",
       "  'Jewelle',\n",
       "  'Cherish',\n",
       "  'Annice'])"
      ]
     },
     "execution_count": 194,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "male_names[:10], female_names[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How many names are given to both boys and girls?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 195,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "365"
      ]
     },
     "execution_count": 195,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "both = [name for name in male_names if name in female_names]\n",
    "len(both)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Okay, that should give us an idea of what we are working."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Feature extraction\n",
    "Back to our goal: make the computer understand names, or really, how to turn any name into a number.\n",
    "\n",
    "In Machine Learning this step is called \"feature extraction\": you extract numbers (the \"feature\") from the thing you really have. In James his fruit-recognition example, that was an image. With our names, it is text.\n",
    "\n",
    "_What you really do is describe the name to the computer in a language it understands: numbers!_\n",
    "\n",
    "Important to remember about feature extraction for machine learning is that we need to be consistent: *If you give it the same name twice, you expect the same numbers each time.*\n",
    "\n",
    "We can think of many ways of turning a name into a number:\n",
    "- The number of letters is has\n",
    "- How often the letter e occurs\n",
    "- The number of vowels it has\n",
    "- Whether it ends with a vowel (0 or 1 is also a number!)\n",
    "- Whether it begins with a vowel\n",
    "- How many letters are repeated\n",
    "\n",
    "Let's just start with the first one, see what we get:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "metadata": {},
   "outputs": [],
   "source": [
    "def extract_length(name):\n",
    "    return len(name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Okay, now to test this. The point of feature extraction is translating the names into a form of information that will help the computer to tell the difference between boys and girls names. We can check whether transforming the names into numbers using the length is helpful by looking at the numbers in a graph. So let's make a histogram: count all the lengths we see, and draw a bar for how often we see them. Green for girls, brown for boys. Easy to remember, it starts with the same letter.\n",
    "\n",
    "The test function is very multi-purpose as we can re-use when we want to test other feature extractors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD4CAYAAAAAczaOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAOpUlEQVR4nO3df6zd9V3H8edLuqn7ESlpV7FtLFkaTLs4RhpAZ8wUZYUtKyZmgSmrE1P/AGVmiWEzsbJlhkTddHFi6lbpIj9CNhaaBceauoSYyOSCDChstmEwWgu9s5Mtkqjo2z/Ot+ZY7q9zf5zvvffzfCQn53ve3885532ae1/n08/3e85NVSFJasMP9N2AJGl8DH1JaoihL0kNMfQlqSGGviQ1ZE3fDcxk3bp1tWXLlr7bkKQV5ZFHHvlOVa2fat+yDv0tW7YwMTHRdxuStKIkeW66fS7vSFJDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQ5b1J3K18uSWjDS+9vpHfKRxcqYvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQ2YN/SSbk3w1yVNJjiS5qaufl+RQkqPd9dquniSfSnIsyeNJLh56rN3d+KNJdi/dy5IkTWUuM/1XgA9V1TbgMuCGJNuAm4HDVbUVONzdBrgS2Npd9gC3weBNAtgLXApcAuw980YhSRqPWUO/qk5W1aPd9veBp4GNwC7gQDfsAHB1t70L+FwNPAScm+R84J3Aoao6XVXfBQ4BOxf11UiSZjTSmn6SLcDbgK8BG6rqZLfrBWBDt70ReH7obse72nT1s59jT5KJJBOTk5OjtCdJmsWcQz/JG4AvAB+squ8N76uqAhblj51W1b6q2lFVO9avX78YDylJ6swp9JO8hkHg31FV93blF7tlG7rrU139BLB56O6butp0dUnSmMzl7J0AnwWerqpPDO06CJw5A2c3cN9Q/f3dWTyXAS91y0APAFckWdsdwL2iq0mSxmTNHMa8HbgOeCLJY13tI8CtwD1JrgeeA97b7bsfuAo4BrwMfACgqk4n+RjwcDfuo1V1elFehSRpTmYN/ar6eyDT7L58ivEF3DDNY+0H9o/SoCRp8fiJXElqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUkFlDP8n+JKeSPDlU+4MkJ5I81l2uGtr34STHknwzyTuH6ju72rEkNy/+S5EkzWYuM/3bgZ1T1D9ZVRd1l/sBkmwDrgG2d/f5iyTnJDkH+DRwJbANuLYbK0kaozWzDaiqB5NsmePj7QLurqr/AL6V5BhwSbfvWFU9A5Dk7m7sUyN3LEmat4Ws6d+Y5PFu+WdtV9sIPD805nhXm67+Kkn2JJlIMjE5ObmA9iRJZ5tv6N8GvBm4CDgJ/MliNVRV+6pqR1XtWL9+/WI9rCSJOSzvTKWqXjyzneSvgC91N08Am4eGbupqzFCXJI3JvGb6Sc4fuvlLwJkzew4C1yT5wSQXAFuBfwQeBrYmuSDJaxkc7D04/7YlSfMx60w/yV3AO4B1SY4De4F3JLkIKOBZ4DcBqupIknsYHKB9Bbihqv67e5wbgQeAc4D9VXVk0V+NJGlGczl759opyp+dYfzHgY9PUb8fuH+k7iRJi8pP5EpSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0JekhszrL2dJi+XO7dtHvs/7jvinGKT5cqYvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGuIpm6tQbslI42tvLVEnkpYbZ/qS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIbMGvpJ9ic5leTJodp5SQ4lOdpdr+3qSfKpJMeSPJ7k4qH77O7GH02ye2lejiRpJnOZ6d8O7DyrdjNwuKq2Aoe72wBXAlu7yx7gNhi8SQB7gUuBS4C9Z94oJEnjM2voV9WDwOmzyruAA932AeDqofrnauAh4Nwk5wPvBA5V1emq+i5wiFe/kUiSlth81/Q3VNXJbvsFYEO3vRF4fmjc8a42Xf1VkuxJMpFkYnJycp7tSZKmsuADuVVVwKL9Ze2q2ldVO6pqx/r16xfrYSVJzD/0X+yWbeiuT3X1E8DmoXGbutp0dUnSGM039A8CZ87A2Q3cN1R/f3cWz2XAS90y0APAFUnWdgdwr+hqkqQxWjPbgCR3Ae8A1iU5zuAsnFuBe5JcDzwHvLcbfj9wFXAMeBn4AEBVnU7yMeDhbtxHq+rsg8OSpCU2a+hX1bXT7Lp8irEF3DDN4+wH9o/UnSRpUfmJXElqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ1Z03cD6t+d27ePfJ/3HTmyBJ1IWmrO9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSF+IlerRm7JSONrby1RJ9LyZegvEQNI0nK0oOWdJM8meSLJY0kmutp5SQ4lOdpdr+3qSfKpJMeSPJ7k4sV4AZKkuVuMNf2fq6qLqmpHd/tm4HBVbQUOd7cBrgS2dpc9wG2L8NySpBEsxYHcXcCBbvsAcPVQ/XM18BBwbpLzl+D5JUnTWGjoF/CVJI8k2dPVNlTVyW77BWBDt70ReH7ovse72v+TZE+SiSQTk5OTC2xPkjRsoQdyf6aqTiR5E3AoyTeGd1ZVJRnpCGVV7QP2AezYscOjm5K0iBY006+qE931KeCLwCXAi2eWbbrrU93wE8Dmobtv6mqSpDGZd+gneX2SN57ZBq4AngQOAru7YbuB+7rtg8D7u7N4LgNeGloGkiSNwUKWdzYAX0xy5nHurKovJ3kYuCfJ9cBzwHu78fcDVwHHgJeBDyzguSVJ8zDv0K+qZ4C3TlH/V+DyKeoF3DDf55MkLZzfvSNJDTH0Jakhhr4kNcTQl6SG+C2by8Sd27ePfJ/3HTmyBJ1IWs2c6UtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5Ia4oez1Cw/EKcWOdOXpIYY+pLUEENfkhpi6EtSQzyQKy2C3JKRxtfeWqJOpJk505ekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGepz/EL+CStNo505ekhqzqmf6on5K8g21L1IkkLQ/O9CWpIYa+JDVkVS/vSMuVJw2oL4a+tML5DZ8ahcs7ktQQQ1+SGmLoS1JDDH1JaogHcqXGeOZQ28Y+00+yM8k3kxxLcvO4n1+SWjbW0E9yDvBp4EpgG3BtEr/7QJLGZNzLO5cAx6rqGYAkdwO7gKfG3IekRTDy91vdM/ocz6WlxZWq8X1QI8kvAzur6je629cBl1bVjUNj9gB7upsXAt8c8WnWAd9ZhHb7sJJ7B/vv20rufyX3Dsuv/x+vqvVT7Vh2B3Krah+wb773TzJRVTsWsaWxWcm9g/33bSX3v5J7h5XV/7gP5J4ANg/d3tTVJEljMO7QfxjYmuSCJK8FrgEOjrkHSWrWWJd3quqVJDcCDwDnAPurarGP0sx7aWgZWMm9g/33bSX3v5J7hxXU/1gP5EqS+uXXMEhSQwx9SWrIqgj9JJuTfDXJU0mOJLmp757mI8k5Sf4pyZf67mVUSc5N8vkk30jydJKf6runuUryO93PzZNJ7kryQ333NJMk+5OcSvLkUO28JIeSHO2u1/bZ40ym6f+Pup+dx5N8Mcm5ffY4k6n6H9r3oSSVZF0fvc3Fqgh94BXgQ1W1DbgMuGGFfr3DTcDTfTcxT38GfLmqfgJ4KyvkdSTZCPw2sKOq3sLgBINr+u1qVrcDO8+q3QwcrqqtwOHu9nJ1O6/u/xDwlqr6SeCfgQ+Pu6kR3M6r+yfJZuAK4NvjbmgUqyL0q+pkVT3abX+fQeBs7Ler0STZBLwL+EzfvYwqyY8APwt8FqCq/rOq/q3frkayBvjhJGuA1wH/0nM/M6qqB4HTZ5V3AQe67QPA1WNtagRT9V9VX6mqV7qbDzH4DM+yNM2/P8Angd8FlvXZMasi9Icl2QK8Dfhav52M7E8Z/MD8T9+NzMMFwCTw193y1GeSvL7vpuaiqk4Af8xgdnYSeKmqvtJvV/OyoapOdtsvABv6bGaBfh34276bGEWSXcCJqvp6373MZlWFfpI3AF8APlhV3+u7n7lK8m7gVFU90ncv87QGuBi4rareBvw7y3t54f90a9+7GLxx/Rjw+iS/2m9XC1OD87CX9WxzOkl+j8Fy7R199zJXSV4HfAT4/b57mYtVE/pJXsMg8O+oqnv77mdEbwfek+RZ4G7g55P8Tb8tjeQ4cLyqzvzv6vMM3gRWgl8AvlVVk1X1X8C9wE/33NN8vJjkfIDu+lTP/Ywsya8B7wZ+pVbWB4jezGDS8PXud3gT8GiSH+21q2msitBPEgbryU9X1Sf67mdUVfXhqtpUVVsYHET8u6paMbPNqnoBeD7JhV3pclbO12V/G7gsyeu6n6PLWSEHoc9yENjdbe8G7uuxl5El2clgefM9VfVy3/2MoqqeqKo3VdWW7nf4OHBx93ux7KyK0GcwU76OwQz5se5yVd9NNea3gDuSPA5cBPxhz/3MSfe/k88DjwJPMPidWNYfqU9yF/APwIVJjie5HrgV+MUkRxn87+XWPnucyTT9/znwRuBQ9/v7l702OYNp+l8x/BoGSWrIapnpS5LmwNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDflfhG+WHOWJgAEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "def test(feature_extractor):\n",
    "    plt.hist([\n",
    "        [feature_extractor(name) for name in female_names],\n",
    "        [feature_extractor(name) for name in male_names]\n",
    "    ], label=['girls', 'boys'], color=['green', 'brown'])\n",
    "\n",
    "test(extract_length)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that there are many more girls names, if you add all their bars together you get a far longer bar than if you add all the boys bars. Let's fix that!\n",
    "\n",
    "Instead of printing the number of times we count a number, let's print the proportion of names with that number. So 3 out of 6 and 4 out of 8 get bars of the same size. Luckily no extra math is necessary for that, matplotlib has a `density=True` option that does this for us. Let's see what the graph looks like if we add that."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAASxUlEQVR4nO3df6zd913f8edr9pyu7Sjpcsc224ndYjpsyhJ0cdgqMokmqbsiO3+kws2KXC3ICopHtzCNdEXJasQU2qkDCUNjtV4riGtCyrSryV2I2sKEIMU3P9pynXlx3C6xFxZTZ3RbuyRO3vvjfINOLte+3+N7r8+9Hz8f0tX9fn4dv4917+t+7/fXTVUhSWrXXxl3AZKkpWXQS1LjDHpJapxBL0mNM+glqXGrx13AbFdccUVt2LBh3GVI0oryyCOP/FlVTcw11ivok2wDfgVYBXyyqu6ZNX4bcDvwMvB/gN1VdTTJBuAJ4Fg39eGquu18/9aGDRuYnp7uU5YkqZPkv59rbN6gT7IK2AfcAJwEjiSZqqqjQ9MOVtUnuvnbgY8D27qxp6rq6gstXpK0MH2O0W8FjlfViap6ETgE7BieUFXfGmq+AfAuLElaJvoE/VrgmaH2ya7vNZLcnuQp4KPAzwwNbUzyWJLfT/KjC6pWkjSyRbvqpqr2VdVbgZ8Dfr7rfha4sqquAe4ADib5rtlrk+xOMp1k+vTp04tVkiSJfkF/Clg/1F7X9Z3LIeAmgKp6oaq+2W0/AjwFfN/sBVW1v6omq2pyYmLOk8aSpAvUJ+iPAJuSbEyyBtgJTA1PSLJpqPke4Mmuf6I7mUuStwCbgBOLUbgkqZ95r7qpqrNJ9gAPMri88kBVzSTZC0xX1RSwJ8n1wEvA88Cubvl1wN4kLwGvALdV1ZmleCOSpLlluT2meHJysryOXpJGk+SRqpqca8xHIEhS45bdIxC08uQjGWl+3b28fouUWucevSQ1zj16XXQHt2wZec0tMzNLUIl0aXCPXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS43oFfZJtSY4lOZ7kzjnGb0vytSSPJ/mDJJuHxj7UrTuW5F2LWbwkaX7zBn2SVcA+4N3AZuB9w0HeOVhVb6+qq4GPAh/v1m4GdgJbgG3Ar3WvJ0m6SPrs0W8FjlfViap6ETgE7BieUFXfGmq+AahuewdwqKpeqKqvA8e715MkXSSre8xZCzwz1D4JXDt7UpLbgTuANcCPDa19eNbatXOs3Q3sBrjyyiv71C1J6mnRTsZW1b6qeivwc8DPj7h2f1VNVtXkxMTEYpUkSaJf0J8C1g+113V953IIuOkC10qSFlmfoD8CbEqyMckaBidXp4YnJNk01HwP8GS3PQXsTHJZko3AJuCPF162JKmveY/RV9XZJHuAB4FVwIGqmkmyF5iuqilgT5LrgZeA54Fd3dqZJPcDR4GzwO1V9fISvRdJ0hz6nIylqg4Dh2f13TW0/cHzrP1F4BcvtEBJ0sJ4Z6wkNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS43oFfZJtSY4lOZ7kzjnG70hyNMlXk3whyVVDYy8nebz7mFrM4iVJ81s934Qkq4B9wA3ASeBIkqmqOjo07TFgsqq+neSngY8CP9GNfaeqrl7kuiVJPfXZo98KHK+qE1X1InAI2DE8oaq+VFXf7poPA+sWt0xJ0oXqE/RrgWeG2ie7vnO5Ffj8UPt1SaaTPJzkpguoUZK0APMeuhlFkvcDk8A/HOq+qqpOJXkL8MUkX6uqp2at2w3sBrjyyisXsyRJuuT12aM/Bawfaq/r+l4jyfXAh4HtVfXCq/1Vdar7fAL4PeCa2Wuran9VTVbV5MTExEhvQJJ0fn2C/giwKcnGJGuAncBrrp5Jcg1wL4OQf26o//Ikl3XbVwDvAIZP4kqSlti8h26q6mySPcCDwCrgQFXNJNkLTFfVFPAx4I3AbycBeLqqtgPfD9yb5BUGP1TumXW1jiRpifU6Rl9Vh4HDs/ruGtq+/hzr/hB4+0IKlCQtjHfGSlLjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxvUK+iTbkhxLcjzJnXOM35HkaJKvJvlCkquGxnYlebL72LWYxUuS5jdv0CdZBewD3g1sBt6XZPOsaY8Bk1X1g8ADwEe7tW8G7gauBbYCdye5fPHKlyTNp88e/VbgeFWdqKoXgUPAjuEJVfWlqvp213wYWNdtvwt4qKrOVNXzwEPAtsUpXZLUR5+gXws8M9Q+2fWdy63A50dZm2R3kukk06dPn+5RkiSpr0U9GZvk/cAk8LFR1lXV/qqarKrJiYmJxSxJki55fYL+FLB+qL2u63uNJNcDHwa2V9ULo6yVJC2dPkF/BNiUZGOSNcBOYGp4QpJrgHsZhPxzQ0MPAjcmubw7CXtj1ydJukhWzzehqs4m2cMgoFcBB6pqJsleYLqqphgcqnkj8NtJAJ6uqu1VdSbJLzD4YQGwt6rOLMk7kSTNad6gB6iqw8DhWX13DW1ff561B4ADF1qgJGlhvDNWkhpn0EtS43odutHyl49kpPl1dy1RJZKWG/foJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY3zhqlL1MEtW0Zec8vMzBJUImmpuUcvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mN6xX0SbYlOZbkeJI75xi/LsmjSc4muXnW2MtJHu8+pharcElSP/M+AiHJKmAfcANwEjiSZKqqjg5Nexr4APAv5niJ71TV1YtQqyTpAvR51s1W4HhVnQBIcgjYAfxF0FfVN7qxV5agRknSAvQ5dLMWeGaofbLr6+t1SaaTPJzkprkmJNndzZk+ffr0CC8tSZrPxTgZe1VVTQK3AL+c5K2zJ1TV/qqarKrJiYmJi1CSJF06+gT9KWD9UHtd19dLVZ3qPp8Afg+4ZoT6JEkL1CfojwCbkmxMsgbYCfS6eibJ5Uku67avAN7B0LF9SdLSmzfoq+ossAd4EHgCuL+qZpLsTbIdIMkPJzkJvBe4N8mrf6Hi+4HpJF8BvgTcM+tqHUnSEuv1F6aq6jBweFbfXUPbRxgc0pm97g+Bty+wRknSAnhnrCQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJalyvoE+yLcmxJMeT3DnH+HVJHk1yNsnNs8Z2JXmy+9i1WIVLkvqZN+iTrAL2Ae8GNgPvS7J51rSngQ8AB2etfTNwN3AtsBW4O8nlCy9bktTX6h5ztgLHq+oEQJJDwA7g6KsTquob3dgrs9a+C3ioqs504w8B24DPLrjyBhzcsmXkNbfMzCxBJZJa1ufQzVrgmaH2ya6vj15rk+xOMp1k+vTp0z1fWpLUx7I4GVtV+6tqsqomJyYmxl2OJDWlT9CfAtYPtdd1fX0sZK0kaRH0CfojwKYkG5OsAXYCUz1f/0HgxiSXdydhb+z6JEkXybxBX1VngT0MAvoJ4P6qmkmyN8l2gCQ/nOQk8F7g3iQz3dozwC8w+GFxBNj76olZSdLF0eeqG6rqMHB4Vt9dQ9tHGByWmWvtAeDAAmqUJC3AsjgZK0laOga9JDXOoJekxvU6Ri8tV/lIRppfd9cSVSItXwb9Iho1dO5j9iODJGnxeehGkhpn0EtS4wx6SWqcx+h1SfHR0LoUuUcvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMb1Cvok25IcS3I8yZ1zjF+W5Le68S8n2dD1b0jynSSPdx+fWNzyJUnzmffplUlWAfuAG4CTwJEkU1V1dGjarcDzVfW9SXYCvwT8RDf2VFVdvch1S5J66rNHvxU4XlUnqupF4BCwY9acHcBnuu0HgHcmGe3v6kmSlkSfoF8LPDPUPtn1zTmnqs4Cfw78jW5sY5LHkvx+kh+d6x9IsjvJdJLp06dPj/QGJEnnt9QnY58Frqyqa4A7gINJvmv2pKraX1WTVTU5MTGxxCVJ0qWlT9CfAtYPtdd1fXPOSbIaeBPwzap6oaq+CVBVjwBPAd+30KIlSf31CfojwKYkG5OsAXYCU7PmTAG7uu2bgS9WVSWZ6E7mkuQtwCbgxOKULknqY96rbqrqbJI9wIPAKuBAVc0k2QtMV9UU8CngN5IcB84w+GEAcB2wN8lLwCvAbVV1ZineiCRpbr3+OHhVHQYOz+q7a2j7/wHvnWPd54DPLbBGSdIC9Ar6lh3csmXkNbfMzCxBJZK0NHwEgiQ17pLfo5cuVD4y2j2BdXctUSXS+blHL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOK+6kS4S79nQuLhHL0mNM+glqXEGvSQ1rrlj9KPerXgfm5eoEklaHtyjl6TGGfSS1DiDXpIa19wxeulS4JMzNQr36CWpce7RS5cA78q9tLlHL0mNM+glqXEGvSQ1rlfQJ9mW5FiS40nunGP8siS/1Y1/OcmGobEPdf3Hkrxr8UqXJPUxb9AnWQXsA94NbAbel2T2cwNuBZ6vqu8F/h3wS93azcBOYAuwDfi17vUkSRdJn6tutgLHq+oEQJJDwA7g6NCcHcC/7rYfAH41Sbr+Q1X1AvD1JMe71/ujxSlf0sU28vOk7h/9eVJe8bO4UnX+GymS3Axsq6qf6to/CVxbVXuG5vxJN+dk134KuJZB+D9cVb/Z9X8K+HxVPTDr39gN7O6abwOOjfg+rgD+bMQ1y8VKrh2sf9xWcv0ruXZYfvVfVVUTcw0si+voq2o/sP9C1yeZrqrJRSzpolnJtYP1j9tKrn8l1w4rq/4+J2NPAeuH2uu6vjnnJFkNvAn4Zs+1kqQl1CfojwCbkmxMsobBydWpWXOmgF3d9s3AF2twTGgK2NldlbMR2AT88eKULknqY95DN1V1Nske4EFgFXCgqmaS7AWmq2oK+BTwG93J1jMMfhjQzbufwYnbs8DtVfXyEryPCz7sswys5NrB+sdtJde/kmuHFVT/vCdjJUkrm3fGSlLjDHpJatyKDfok65N8KcnRJDNJPjjumi5EklVJHkvyn8Zdy6iSfHeSB5L81yRPJPn7466pryT/vPu6+ZMkn03yunHXdD5JDiR5rrtn5dW+Nyd5KMmT3efLx1nj+Zyj/o91XztfTfIfknz3OGs8n7nqHxr72SSV5Ipx1NbHig16Bid3f7aqNgM/Atw+x6MZVoIPAk+Mu4gL9CvAf66qvwv8PVbI+0iyFvgZYLKqfoDBRQY7x1vVvD7N4DEiw+4EvlBVm4AvdO3l6tP85fofAn6gqn4Q+G/Ahy52USP4NH+5fpKsB24Enr7YBY1ixQZ9VT1bVY922/+bQcisHW9Vo0myDngP8Mlx1zKqJG8CrmNwxRVV9WJV/a/xVjWS1cBf6+77eD3wP8Zcz3lV1X9hcEXbsB3AZ7rtzwA3XdSiRjBX/VX1u1V1tms+zOA+m2XpHP//MHi2178ElvVVLSs26Id1T8u8BvjyeCsZ2S8z+CJ5ZdyFXICNwGng33eHnj6Z5A3jLqqPqjoF/FsGe2HPAn9eVb873qouyPdU1bPd9p8C3zPOYhbonwCfH3cRo0iyAzhVVV8Zdy3zWfFBn+SNwOeAf1ZV3xp3PX0l+XHguap6ZNy1XKDVwA8Bv15V1wD/l+V96OAvdMeydzD4YfV3gDckef94q1qY7gbFZb1XeS5JPszgUOx9466lrySvB/4VcNe4a+ljRQd9kr/KIOTvq6rfGXc9I3oHsD3JN4BDwI8l+c3xljSSk8DJqnr1t6gHGAT/SnA98PWqOl1VLwG/A/yDMdd0If5nkr8N0H1+bsz1jCzJB4AfB/5xraybet7KYEfhK9338Drg0SR/a6xVncOKDfruMcifAp6oqo+Pu55RVdWHqmpdVW1gcCLwi1W1YvYqq+pPgWeSvK3reievfXT1cvY08CNJXt99Hb2TFXIieZbhR4/sAv7jGGsZWZJtDA5dbq+qb4+7nlFU1deq6m9W1Ybue/gk8EPd98Wys2KDnsEe8U8y2BN+vPv4R+Mu6hLzT4H7knwVuBr4N2Oup5fut5AHgEeBrzH4PljWt7Mn+SyDv+PwtiQnk9wK3APckORJBr+l3DPOGs/nHPX/KvDXgYe6799PjLXI8zhH/SuGj0CQpMat5D16SVIPBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklq3P8HkdxPs2spnHIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def test(feature_extractor):\n",
    "    plt.hist([\n",
    "        [feature_extractor(name) for name in female_names],\n",
    "        [feature_extractor(name) for name in male_names]\n",
    "    ], label=['girls', 'boys'], density=True, color=['green', 'brown'])\n",
    "\n",
    "test(extract_length)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Well that doesn't work great, they are about the same. Both groups of names are on average 6 letters long."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try our second attempt: how often does the letter 'e' occur?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAPKElEQVR4nO3df4xld13G8ffDbgsmIE3YMTa7W7bGxbiLRepkramRKpBsq9n9A6K7FbCkZRO1igE1RU1Lyz8iCSpSrGtp+CFtqZWQFbdpiJQ0EVs75UfpbC0ZCtqtJDsULBqUuvrxj3sXr9OZuWdmz8yd/fb9Sia555zv3PPku3ufOXPuOXdSVUiSznzPmXQASVI/LHRJaoSFLkmNsNAlqREWuiQ1YvOkdrxly5basWPHpHYvSWekBx988OtVNbXYtokV+o4dO5iZmZnU7iXpjJTkn5ba5ikXSWqEhS5JjbDQJakRFrokNcJCl6RGjC30JLckOZHk4SW2/2KSh5J8Mclnkrys/5iSpHG6HKF/ANi7zPavAK+oqh8B3gEc7iGXJGmFxl6HXlX3JtmxzPbPjCzeB2w7/ViSpJXq+xz6lcBdPT+nJKmD3u4UTfLTDAr9J5cZcwg4BHDeeef1tetObt29e0XjL5+dXaMkkrQ2ejlCT3IBcDOwv6qeXGpcVR2uqumqmp6aWvSjCCRJq3TahZ7kPOBjwOur6kunH0mStBpjT7kkuQ24BNiS5DhwHXAWQFXdBFwLvAh4XxKAk1U1vVaBJUmL63KVy8Ex268CruotkSRpVbxTVJIaYaFLUiMsdElqhIUuSY2w0CWpERa6JDXCQpekRljoktQIC12SGmGhS1IjLHRJaoSFLkmNsNAlqREWuiQ1wkKXpEZY6JLUCAtdkhphoUtSIyx0SWrE2L8pqjPbrbt3r2j85bOza5RE0lrzCF2SGmGhS1IjLHRJasQZeQ4912fF3/MRdq1BEknaOM7IQn82W+kPM3+QSc8ennKRpEaMLfQktyQ5keThJbYnyXuSzCV5KMmF/ceUJI3T5Qj9A8DeZbZfCuwcfh0C/vT0Y0mSVmpsoVfVvcA3lhmyH/hQDdwHnJPk3L4CSpK66eMc+lbg8ZHl48N1z5DkUJKZJDPz8/M97FqSdMq6vilaVYerarqqpqemptZz15LUvD4K/Qlg+8jytuE6SdI66qPQjwBvGF7tchHwVFV9rYfnlSStwNgbi5LcBlwCbElyHLgOOAugqm4CjgKXAXPAt4E3rlVYSdLSxhZ6VR0cs72AX+0tkSRpVbxTVJIaYaFLUiMsdElqhIUuSY2w0CWpERa6JDXCQpekRljoktQIC12SGmGhS1IjLHRJaoSFLkmNsNAlqREWuiQ1wkKXpEZY6JLUCAtdkhphoUtSIyx0SWqEhS5JjbDQJakRFrokNcJCl6RGWOiS1AgLXZIaYaFLUiM6FXqSvUkeTTKX5JpFtp+X5J4kn0vyUJLL+o8qSVrO2EJPsgm4EbgU2AUcTLJrwbDfA+6oqpcDB4D39R1UkrS8Lkfoe4C5qnqsqp4Gbgf2LxhTwPcOH78Q+Jf+IkqSuuhS6FuBx0eWjw/XjXo78Lokx4GjwK8t9kRJDiWZSTIzPz+/iriSpKX09aboQeADVbUNuAz4cJJnPHdVHa6q6aqanpqa6mnXkiToVuhPANtHlrcN1426ErgDoKr+HngesKWPgJKkbroU+gPAziTnJzmbwZueRxaM+WfglQBJfphBoXtORZLW0dhCr6qTwNXA3cAjDK5mmU1yQ5J9w2FvBd6U5AvAbcAVVVVrFVqS9EybuwyqqqMM3uwcXXftyONjwMX9RpMkrYR3ikpSIyx0SWqEhS5JjbDQJakRFrokNcJCl6RGWOiS1AgLXZIaYaFLUiMsdElqhIUuSY2w0CWpERa6JDXCQpekRljoktQIC12SGmGhS1IjLHRJaoSFLkmNsNAlqREWuiQ1wkKXpEZY6JLUCAtdkhphoUtSIzoVepK9SR5NMpfkmiXG/HySY0lmk9zab0xJ0jibxw1Isgm4EXg1cBx4IMmRqjo2MmYn8Dbg4qr6ZpLvW6vAkqTFdTlC3wPMVdVjVfU0cDuwf8GYNwE3VtU3AarqRL8xJUnjdCn0rcDjI8vHh+tGvQR4SZK/S3Jfkr2LPVGSQ0lmkszMz8+vLrEkaVF9vSm6GdgJXAIcBP48yTkLB1XV4aqarqrpqampnnYtSYJuhf4EsH1kedtw3ajjwJGq+q+q+grwJQYFL0laJ10K/QFgZ5Lzk5wNHACOLBjzcQZH5yTZwuAUzGM95pQkjTG20KvqJHA1cDfwCHBHVc0muSHJvuGwu4EnkxwD7gF+q6qeXKvQkqRnGnvZIkBVHQWOLlh37cjjAt4y/JIkTYB3ikpSIyx0SWqEhS5JjbDQJakRFrokNcJCl6RGWOiS1AgLXZIaYaFLUiMsdElqhIUuSY2w0CWpERa6JDXCQpekRljoktQIC12SGmGhS1IjLHRJaoSFLkmNsNAlqREWuiQ1wkKXpEZY6JLUCAtdkhphoUtSIyx0SWpEp0JPsjfJo0nmklyzzLjXJKkk0/1FlCR1MbbQk2wCbgQuBXYBB5PsWmTcC4A3A/f3HVKSNF6XI/Q9wFxVPVZVTwO3A/sXGfcO4J3Af/aYT5LUUZdC3wo8PrJ8fLjuu5JcCGyvqr9Z7omSHEoyk2Rmfn5+xWElSUs77TdFkzwHeDfw1nFjq+pwVU1X1fTU1NTp7lqSNKJLoT8BbB9Z3jZcd8oLgJcCn07yVeAi4IhvjErS+trcYcwDwM4k5zMo8gPA5ac2VtVTwJZTy0k+DfxmVc30G1WajFyfFY2v62qNkkjLG3uEXlUngauBu4FHgDuqajbJDUn2rXVASVI3XY7QqaqjwNEF665dYuwlpx9LkrRS3ikqSY2w0CWpERa6JDXCQpekRljoktQIC12SGmGhS1IjLHRJaoSFLkmNsNAlqREWuiQ1wkKXpEZY6JLUCAtdkhphoUtSIyx0SWqEhS5JjbDQJakRFrokNcJCl6RGWOiS1AgLXZIaYaFLUiMsdElqxOZJB5Bac+vu3Sv+nstnZ9cgiZ5tOh2hJ9mb5NEkc0muWWT7W5IcS/JQkr9N8uL+o0qSljO20JNsAm4ELgV2AQeT7Fow7HPAdFVdANwJ/EHfQSVJy+tyhL4HmKuqx6rqaeB2YP/ogKq6p6q+PVy8D9jWb0xJ0jhdCn0r8PjI8vHhuqVcCdy12IYkh5LMJJmZn5/vnlKSNFavV7kkeR0wDbxrse1Vdbiqpqtqempqqs9dS9KzXperXJ4Ato8sbxuu+3+SvAr4XeAVVfWdfuJJkrrqcoT+ALAzyflJzgYOAEdGByR5OfBnwL6qOtF/TEnSOGMLvapOAlcDdwOPAHdU1WySG5LsGw57F/B84C+TfD7JkSWeTpK0RjrdWFRVR4GjC9ZdO/L4VT3nkiStkLf+S1IjLHRJaoSFLkmNsNAlqREWuiQ1wkKXpEZY6JLUCAtdkhphoUtSIyx0SWqEhS5JjbDQJakRFrokNcJCl6RGWOiS1AgLXZIaYaFLUiMsdElqhIUuSY2w0CWpEZ3+SLSkZ59cnxWNr+tqjZKoK4/QJakRFrokNcJCl6RGWOiS1AgLXZIa0ekqlyR7gT8GNgE3V9XvL9j+XOBDwI8BTwK/UFVf7TeqpI3s1t27V/w9l8/OrkGSZ6+xR+hJNgE3ApcCu4CDSXYtGHYl8M2q+kHgD4F39h1UkrS8Lkfoe4C5qnoMIMntwH7g2MiY/cDbh4/vBN6bJFXlhamSzihn8vX3Gde5SV4L7K2qq4bLrwd+vKquHhnz8HDM8eHyl4djvr7guQ4Bh4aLPwQ8uoKsW4Cvjx01GWZbHbOt3EbNBWZbrZVme3FVTS22YV3vFK2qw8Dh1Xxvkpmqmu45Ui/MtjpmW7mNmgvMtlp9ZutylcsTwPaR5W3DdYuOSbIZeCGDN0clSeukS6E/AOxMcn6Ss4EDwJEFY44AvzR8/FrgU54/l6T1NfaUS1WdTHI1cDeDyxZvqarZJDcAM1V1BHg/8OEkc8A3GJR+31Z1qmadmG11zLZyGzUXmG21ess29k1RSdKZwTtFJakRFrokNWLDFXqSvUkeTTKX5JpFtj83yUeH2+9PsmMDZbsiyXySzw+/rlqnXLckOTG8H2Cx7UnynmHuh5JcuB65Oma7JMlTI3N27Trl2p7kniTHkswmefMiYyYybx2zTWrenpfkH5J8YZjt+kXGTOQ12jHbRF6jw31vSvK5JJ9YZFs/c1ZVG+aLwZuuXwZ+ADgb+AKwa8GYXwFuGj4+AHx0A2W7AnjvBObtp4ALgYeX2H4ZcBcQ4CLg/g2U7RLgExOYs3OBC4ePXwB8aZF/z4nMW8dsk5q3AM8fPj4LuB+4aMGYSb1Gu2SbyGt0uO+3ALcu9u/W15xttCP0737MQFU9DZz6mIFR+4EPDh/fCbwyycru1V27bBNRVfcyuLpoKfuBD9XAfcA5Sc7dINkmoqq+VlWfHT7+N+ARYOuCYROZt47ZJmI4F/8+XDxr+LXwyoqJvEY7ZpuIJNuAnwVuXmJIL3O20Qp9K/D4yPJxnvkf+btjquok8BTwog2SDeA1w1/P70yyfZHtk9A1+6T8xPDX5LuSrPwj+07T8NfblzM4ohs18XlbJhtMaN6Gpw4+D5wAPllVS87bOr9Gu2SDybxG/wj4beB/ltjey5xttEI/0/01sKOqLgA+yf/9xNXSPsvgsyleBvwJ8PH13HmS5wN/BfxGVX1rPfc9zphsE5u3qvrvqvpRBneN70ny0vXa9zgdsq37azTJzwEnqurBtd7XRiv0jfwxA2OzVdWTVfWd4eLNDD4ffiPoMq8TUVXfOvVrclUdBc5KsmU99p3kLAaF+ZGq+tgiQyY2b+OyTXLeRjL8K3APsHfBpol/FMhS2Sb0Gr0Y2JfkqwxO1f5Mkr9YMKaXOdtohb6RP2ZgbLYF51f3MTj3uREcAd4wvGrjIuCpqvrapEMBJPn+U+cKk+xh8H9yzV/8w32+H3ikqt69xLCJzFuXbBOct6kk5wwffw/wauAfFwybyGu0S7ZJvEar6m1Vta2qdjDojU9V1esWDOtlztb10xbHqY3zMQOrzfbrSfYBJ4fZrliPbEluY3DVw5Ykx4HrGLwhRFXdBBxlcMXGHPBt4I3rkatjttcCv5zkJPAfwIF1+gF9MfB64IvDc64AvwOcN5JtUvPWJduk5u1c4IMZ/OGb5wB3VNUnNsJrtGO2ibxGF7MWc+at/5LUiI12ykWStEoWuiQ1wkKXpEZY6JLUCAtdkhphoUtSIyx0SWrE/wJArkGE41+kbAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def extract_e(name):\n",
    "    return name.count('e')\n",
    "\n",
    "test(extract_e)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again pretty similar. How about the number of vowels?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 200,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAPkElEQVR4nO3df4ydWV3H8feHlvpjQTDpaDZtoY0WTBcJC2PRYJAga7qBtCSgtgQCBmxMKK6uUbtqumz9RyABTWyMdVkDuqWui5hRRyuRNYpxsbOwgtNSHOtqp9HssCwiGimFr3/MXbjMTuc+07mzt/fs+5VMes95zjzP90nTT07P8+OmqpAkjb+njLoASdJwGOiS1AgDXZIaYaBLUiMMdElqxMZRHXjz5s21ffv2UR1eksbSAw888Nmqmlhu28gCffv27czMzIzq8JI0lpL825W2ueQiSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNGNmTomrfiRtuWNX4183OrlMl0pODM3RJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhrRKdCT7ElyLslcksPLbH9Pkgd7P59J8vnhlypJWsnAB4uSbACOATcB88DpJFNVdeaxMVX1s33j3wbcuA61SpJW0GWGvhuYq6rzVXUJOAnsW2H8AeADwyhOktRdl0DfAlzoa8/3+h4nybOBHcBHrrD9YJKZJDMLCwurrVWStIJhXxTdD9xbVV9ZbmNVHa+qyaqanJiYGPKhJenJrUugXwS29bW39vqWsx+XWyRpJLoE+mlgZ5IdSTaxGNpTSwcl+R7g24G/H26JkqQuBgZ6VV0GDgGngLPAPVU1m+Rokr19Q/cDJ6uq1qdUSdJKOr0PvaqmgeklfUeWtN8+vLIkSavlk6KS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmN6PS2RenJLHdkVePrdt8grdFwhi5JjTDQJakRBrokNcJAl6RGGOiS1IhOgZ5kT5JzSeaSHL7CmB9LcibJbJITwy1TkjTIwNsWk2wAjgE3AfPA6SRTVXWmb8xO4DbgJVX1aJLvWK+CJUnL6zJD3w3MVdX5qroEnAT2LRnzk8CxqnoUoKoeHm6ZkqRBugT6FuBCX3u+19fvOcBzkvxdkvuT7FluR0kOJplJMrOwsHB1FUuSljWsi6IbgZ3Ay4ADwO8keebSQVV1vKomq2pyYmJiSIeWJEG3QL8IbOtrb+319ZsHpqrqy1X1r8BnWAx4SdITpEugnwZ2JtmRZBOwH5haMuaPWZydk2Qzi0sw54dYpyRpgIGBXlWXgUPAKeAscE9VzSY5mmRvb9gp4JEkZ4D7gJ+vqkfWq2hJ0uN1ettiVU0D00v6jvR9LuDW3o8kaQR8UlSSGmGgS1IjDHRJaoSBLkmNMNAlqRF+p6g6We33agLcza51qETSlThDl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJakSnQE+yJ8m5JHNJDi+z/U1JFpI82Pt5y/BLlSStZODrc5NsAI4BNwHzwOkkU1V1ZsnQP6iqQ+tQoySpgy4z9N3AXFWdr6pLwElg3/qWJUlarS6BvgW40Nee7/Ut9Zokn0xyb5Jty+0oycEkM0lmFhYWrqJcSdKVDOui6J8A26vq+cCHgfctN6iqjlfVZFVNTkxMDOnQkiToFugXgf4Z99Ze39dU1SNV9aVe807gRcMpT5LUVZdAPw3sTLIjySZgPzDVPyDJ9X3NvcDZ4ZUoSepi4F0uVXU5ySHgFLABuKuqZpMcBWaqagr46SR7gcvA54A3rWPNkqRlDAx0gKqaBqaX9B3p+3wbcNtwS5MkrYZPikpSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqRKdH/7U2J264YVXjXzc7u06VSGqZM3RJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIzoFepI9Sc4lmUtyeIVxr0lSSSaHV6IkqYuBgZ5kA3AMuBnYBRxIsmuZcU8HbgE+NuwiJUmDdZmh7wbmqup8VV0CTgL7lhn3q8A7gP8bYn2SpI66BPoW4EJfe77X9zVJXghsq6o/W2lHSQ4mmUkys7CwsOpiJUlXtuaLokmeArwb+LlBY6vqeFVNVtXkxMTEWg8tSerTJdAvAtv62lt7fY95OvA84K+TPAR8PzDlhVFJemJ1CfTTwM4kO5JsAvYDU49trKr/qqrNVbW9qrYD9wN7q2pmXSqWJC1r4PvQq+pykkPAKWADcFdVzSY5CsxU1dTKe5CeXHz/vUal0xdcVNU0ML2k78gVxr5s7WVJklbLJ0UlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRnQK9CR7kpxLMpfk8DLbfyrJp5I8mOSjSXYNv1RJ0koGBnqSDcAx4GZgF3BgmcA+UVXfW1UvAN4JvHvolUqSVtRlhr4bmKuq81V1CTgJ7OsfUFVf6GteB9TwSpQkdbGxw5gtwIW+9jzw4qWDkrwVuBXYBLx8uR0lOQgcBHjWs5612lolSSsY2kXRqjpWVd8F/CLwK1cYc7yqJqtqcmJiYliHliTRLdAvAtv62lt7fVdyEnj1WoqSJK1el0A/DexMsiPJJmA/MNU/IMnOvuYrgX8eXomSpC4GrqFX1eUkh4BTwAbgrqqaTXIUmKmqKeBQklcAXwYeBd64nkVLkh6vy0VRqmoamF7Sd6Tv8y1DrkuStEo+KSpJjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEZ0e/dfX5Y6s+nfuxm/kk7T+nKFLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGtEp0JPsSXIuyVySw8tsvzXJmSSfTPJXSZ49/FIlSSsZGOhJNgDHgJuBXcCBJEsfffwEMFlVzwfuBd457EIlSSvrMkPfDcxV1fmqugScBPb1D6iq+6rqf3vN+4Gtwy1TkjRIl0DfAlzoa8/3+q7kzcCfL7chycEkM0lmFhYWulcpSRpoqBdFk7wemATetdz2qjpeVZNVNTkxMTHMQ0vSk16Xty1eBLb1tbf2+r5BklcAvwz8UFV9aTjlSU9uq327Z91e61SJxkGXGfppYGeSHUk2AfuBqf4BSW4EfhvYW1UPD79MSdIgAwO9qi4Dh4BTwFngnqqaTXI0yd7esHcBTwP+MMmDSaausDtJ0jrp9AUXVTUNTC/pO9L3+RVDrkuStEo+KSpJjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1olOgJ9mT5FySuSSHl9n+0iQfT3I5yWuHX6YkaZCBgZ5kA3AMuBnYBRxIsmvJsH8H3gScGHaBkqRuNnYYsxuYq6rzAElOAvuAM48NqKqHetu+ug41SpI66LLksgW40Nee7/WtWpKDSWaSzCwsLFzNLiRJV/CEXhStquNVNVlVkxMTE0/koSWpeV0C/SKwra+9tdcnSbqGdAn008DOJDuSbAL2A1PrW5YkabUGBnpVXQYOAaeAs8A9VTWb5GiSvQBJvi/JPPCjwG8nmV3PoiVJj9flLheqahqYXtJ3pO/zaRaXYiRJI+KTopLUCANdkhrRaclF0ng4ccMNqxr/ulkvd7XEGbokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjxvJti7kjq/6du+/ZtarxvoVO0rgZy0CX1LbVTtrq9lqnSsaLSy6S1IhOM/Qke4DfADYAd1bVry3Z/k3A+4EXAY8AP15VDw23VEla3mq/2APaXFYdOENPsgE4BtwM7AIOJFm6IP1m4NGq+m7gPcA7hl2oJGllXWbou4G5qjoPkOQksA840zdmH/D23ud7gd9MkqpyYUvSWBnn9fsMytwkrwX2VNVbeu03AC+uqkN9Y/6pN2a+1/6X3pjPLtnXQeBgr/lc4Nwqat0MfHbgqPHh+Vy7WjoX8Hyudas9n2dX1cRyG57Qu1yq6jhw/Gp+N8lMVU0OuaSR8XyuXS2dC3g+17phnk+Xu1wuAtv62lt7fcuOSbIReAaLF0clSU+QLoF+GtiZZEeSTcB+YGrJmCngjb3PrwU+4vq5JD2xBi65VNXlJIeAUyzetnhXVc0mOQrMVNUU8F7g95LMAZ9jMfSH7aqWaq5hns+1q6VzAc/nWje08xl4UVSSNB58UlSSGmGgS1IjxiLQk+xJci7JXJLDo65nLZLcleTh3r37Yy3JtiT3JTmTZDbJLaOuaS2SfHOSf0jyj73zuWPUNQ1Dkg1JPpHkT0ddy1oleSjJp5I8mGRm1PWsRZJnJrk3yaeTnE3yA2ve57W+ht579cBngJuAeRbvujlQVWdW/MVrVJKXAl8E3l9Vzxt1PWuR5Hrg+qr6eJKnAw8Arx7jv5sA11XVF5M8FfgocEtV3T/i0tYkya3AJPBtVfWqUdezFkkeAiaXPrQ4jpK8D/jbqrqzdwfht1bV59eyz3GYoX/t1QNVdQl47NUDY6mq/obFO4HGXlX9R1V9vPf5v4GzwJbRVnX1atEXe82n9n6u7RnPAEm2Aq8E7hx1Lfq6JM8AXsriHYJU1aW1hjmMR6BvAS70tecZ49BoVZLtwI3Ax0Zbydr0liceBB4GPlxVY30+wK8DvwB8ddSFDEkBf5nkgd6rRMbVDmAB+N3ectidSa5b607HIdB1jUvyNOCDwM9U1RdGXc9aVNVXquoFLD4RvTvJ2C6LJXkV8HBVPTDqWoboB6vqhSy+/fWtvSXMcbQReCHwW1V1I/A/wJqvD45DoHd59YBGpLfW/EHg7qr6o1HXMyy9//7eB+wZdS1r8BJgb2/d+STw8iS/P9qS1qaqLvb+fBj4EItLsuNoHpjv+x/gvSwG/JqMQ6B3efWARqB3EfG9wNmqeveo61mrJBNJntn7/C0sXoj/9GirunpVdVtVba2q7Sz+u/lIVb1+xGVdtSTX9S6+01ue+BFgLO8Wq6r/BC4keW6v64f5xleSX5Vr/jtFr/TqgRGXddWSfAB4GbA5yTxwe1W9d7RVXbWXAG8APtVbdwb4paqaHmFNa3E98L7enVVPAe6pqrG/1a8h3wl8aHEewUbgRFX9xWhLWpO3AXf3JqrngZ9Y6w6v+dsWJUndjMOSiySpAwNdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNeL/ASCXJxGNrhF1AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def extract_number_of_vowels(name):\n",
    "    return sum(name.count(vowel) for vowel in 'aeiou')\n",
    "\n",
    "test(extract_number_of_vowels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Getting there! At least there is a difference. Let's try whether a name ends with a vowel?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 201,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAMeUlEQVR4nO3db6xkd13H8c+H3VakVmrY0RBauTSBxm2NtJlUCKZKK2RbTPuAxmxJUUj1BhSC0cRgSNTqIx9I0KRRb7CC2pZCpWTDP0Vp00DYhdl2Kd0tNW2tsrWyU5EKGCmFjw/mbHu5md05s8458+2d9yu56cydc+9+fzt33z333HPuOIkAAHU9Z9kDAABOjlADQHGEGgCKI9QAUByhBoDidnbxSXft2pW1tbUuPjUAbEsHDx58PMlg2mOdhHptbU2j0aiLTw0A25Ltfz3RYxz6AIDiCDUAFEeoAaA4Qg0AxRFqACiOUANAcYQaAIoj1ABQHKEGgOI6uTKxTzeff/7cH/OGw4c7mAQAusEeNQAUR6gBoDhCDQDFEWoAKI5QA0BxhBoAiiPUAFAcoQaA4gg1ABRHqAGguJmXkNs+T9Ktm951rqTfTfKezqYCgAXz9Z77Y/J76WCS+c0MdZIHJL1ckmzvkPSopNs7ngsA0Jj30Mdlkh5KcsKXNQcALNa8od4r6ZYuBgEATNc61LZPl3SlpA+d4PF12yPbo/F4vKj5AGDlzbNHfbmku5N8ddqDSTaSDJMMB4PBYqYDAMwV6mvEYQ8A6F2rUNs+Q9JrJH2423EAAFu1eimuJN+S9IKOZwEATMGViQBQHKEGgOIINQAUR6gBoDhCDQDFEWoAKI5QA0BxhBoAiiPUAFAcoQaA4gg1ABRHqAGgOEINAMURagAojlADQHGEGgCKI9QAUByhBoDiCDUAFNf2xW3Psn2b7S/bvt/2K7seDAAw0erFbSX9iaRPJrna9umSntfhTACATWaG2vbzJV0i6U2SlORJSU92OxYA4Lg2hz5eImks6a9s32P7vbbP2LqR7XXbI9uj8Xi88EEBYFW1CfVOSRdJ+rMkF0r6lqR3bt0oyUaSYZLhYDBY8JgAsLrahPqopKNJDjT3b9Mk3ACAHswMdZL/kPQV2+c177pM0pFOpwIAPK3tWR9vl3RTc8bHw5Le3N1IAIDNWoU6ySFJw45nAQBMwZWJAFAcoQaA4gg1ABRHqAGgOEINAMURagAojlADQHGEGgCKI9QAUByhBoDiCDUAFEeoAaA4Qg0AxRFqACiOUANAcYQaAIoj1ABQHKEGgOJavRSX7UckfUPSdyU9lYSX5QKAnrR9cVtJenWSxzubBAAwFYc+AKC4tqGOpH+wfdD2+rQNbK/bHtkejcfjxU0IACuubah/JslFki6X9Ou2L9m6QZKNJMMkw8FgsNAhAWCVtQp1kkeb/x6TdLuki7scCgDwjJmhtn2G7TOP35b0Wkn3dT0YAGCizVkfPybpdtvHt785ySc7nQoA8LSZoU7ysKSf6mEWAMAUnJ4HAMURagAojlADQHGEGgCKI9QAUByhBoDiCDUAFEeoAaA4Qg0AxRFqACiOUANAcYQaAIoj1ABQHKEGgOIINQAUR6gBoDhCDQDFEWoAKI5QA0BxrUNte4fte2x/tMuBAADfb5496ndIur+rQQAA07UKte2zJb1O0nu7HQcAsFXbPer3SPptSd870Qa2122PbI/G4/FChgMAtAi17V+QdCzJwZNtl2QjyTDJcDAYLGxAAFh1bfaoXyXpStuPSPqApEtt/22nUwEAnjYz1El+J8nZSdYk7ZX06STXdj4ZAEAS51EDQHk759k4yZ2S7uxkEgDAVOxRA0BxhBoAiiPUAFAcoQaA4gg1ABRHqAGgOEINAMURagAojlADQHGEGgCKI9QAUByhBoDiCDUAFEeoAaA4Qg0AxRFqACiOUANAcYQaAIqbGWrbz7X9edtftH3Y9vV9DAYAmGjzmonflnRpkm/aPk3SZ2x/Isn+jmcDAKhFqJNE0jebu6c1b+lyKADAM1odo7a9w/YhScckfSrJgSnbrNse2R6Nx+NFzwkAK6tVqJN8N8nLJZ0t6WLbF0zZZiPJMMlwMBgsek4AWFlznfWR5OuS7pC0p5txAABbtTnrY2D7rOb2D0p6jaQvdz0YAGCizVkfL5T0fts7NAn7B5N8tKuBfL3n2v4m7e5oEgCooc1ZH/dKurCHWQAAU3BlIgAUR6gBoDhCDQDFEWoAKI5QA0BxhBoAiiPUAFAcoQaA4gg1ABTX5hJyAFhJN59//lzbv+Hw4U7mYI8aAIoj1ABQHKEGgOIINQAUR6gBoDhCDQDFEWoAKI5QA0BxhBoAimvzKuTn2L7D9hHbh22/o4/BAAATbS4hf0rSbyW52/aZkg7a/lSSIx3PBgBQiz3qJI8lubu5/Q1J90t6UdeDAQAm5jpGbXtN0oWSDkx5bN32yPZoPB4vZjoAQPtQ2/4hSX8n6TeS/PfWx5NsJBkmGQ4Gg0XOCAArrVWobZ+mSaRvSvLhbkcCAGzW5qwPS/pLSfcneXf3IwEANmuzR/0qSW+UdKntQ83bFR3PBQBozDw9L8lnJLmHWQAAU3BlIgAUR6gBoDhCDQDFEWoAKI5QA0BxhBoAiiPUAFAcoQaA4gg1ABRHqAGgOEINAMURagAojlADQHGEGgCKI9QAUByhBoDiCDUAFEeoAaC4Ni9ue6PtY7bv62MgAMD3a7NH/T5JezqeAwBwAjNDneQuSV/rYRYAwBQLO0Zte932yPZoPB4v6tMCwMpbWKiTbCQZJhkOBoNFfVoAWHmc9QEAxRFqACiuzel5t0j6nKTzbB+1fV33YwEAjts5a4Mk1/QxCABgOg59AEBxhBoAiiPUAFAcoQaA4gg1ABRHqAGgOEINAMURagAojlADQHGEGgCKI9QAUByhBoDiCDUAFEeoAaA4Qg0AxRFqACiOUANAcYQaAIoj1ABQXKtQ295j+wHbD9p+Z9dDAQCe0eZVyHdIukHS5ZJ2S7rG9u6uBwMATLTZo75Y0oNJHk7ypKQPSLqq27EAAMc5yck3sK+WtCfJrzT33yjpp5O8bct265LWm7vnSXpgjjl2SXp8ju23C9a9Wlj3apl33S9OMpj2wM7FzCMl2ZC0cSofa3uUZLioWZ4tWPdqYd2rZZHrbnPo41FJ52y6f3bzPgBAD9qE+guSXmr7JbZPl7RX0r5uxwIAHDfz0EeSp2y/TdLfS9oh6cYkhxc8xykdMtkGWPdqYd2rZWHrnvnDRADAcnFlIgAUR6gBoLjeQj3rMnTbP2D71ubxA7bX+pqtSy3W/Zu2j9i+1/Y/2X7xMubsQttfPWD79bZje1ucwtVm3bZ/sXneD9u+ue8Zu9Dia/3Hbd9h+57m6/2KZcy5SLZvtH3M9n0neNy2/7T5O7nX9kWn9Acl6fxNkx9CPiTpXEmnS/qipN1btvk1SX/e3N4r6dY+Ziuw7ldLel5z+63bYd1t195sd6akuyTtlzRc9tw9PecvlXSPpB9p7v/osufuad0bkt7a3N4t6ZFlz72AdV8i6SJJ953g8SskfUKSJb1C0oFT+XP62qNucxn6VZLe39y+TdJltt3TfF2Zue4kdyT5n+bufk3OU98O2v7qgT+U9EeS/rfP4TrUZt2/KumGJP8lSUmO9TxjF9qsO5J+uLn9fEn/3uN8nUhyl6SvnWSTqyT9dSb2SzrL9gvn/XP6CvWLJH1l0/2jzfumbpPkKUlPSHpBL9N1p826N7tOk//7bgcz1958G3hOko/1OVjH2jznL5P0Mtuftb3f9p7eputOm3X/vqRrbR+V9HFJb+9ntKWatwFTLewScvz/2L5W0lDSzy57lj7Yfo6kd0t605JHWYadmhz++DlNvoO6y/ZPJvn6Uqfq3jWS3pfkj22/UtLf2L4gyfeWPVh1fe1Rt7kM/eltbO/U5Fuj/+xluu60uvze9s9LepekK5N8u6fZujZr7WdKukDSnbYf0eT43b5t8APFNs/5UUn7knwnyb9I+mdNwv1s1mbd10n6oCQl+Zyk52ryi4u2s4X8Co6+Qt3mMvR9kn65uX21pE+nORr/LDZz3bYvlPQXmkR6OxyrPO6ka0/yRJJdSdaSrGlyfP7KJKPljLswbb7WP6LJ3rRs79LkUMjDfQ7ZgTbr/jdJl0mS7Z/QJNTjXqfs3z5Jv9Sc/fEKSU8keWzuz9LjT0ev0GTP4SFJ72re9wea/OOUJk/ahyQ9KOnzks5d9k90e1r3P0r6qqRDzdu+Zc/c19q3bHuntsFZHy2fc2ty2OeIpC9J2rvsmXta925Jn9XkjJBDkl677JkXsOZbJD0m6TuafKd0naS3SHrLpuf6hubv5Eun+jXOJeQAUBxXJgJAcYQaAIoj1ABQHKEGgOIINQAUR6gBoDhCDQDF/R8J3REYEW0tGQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def extract_ends_with_vowel(name):\n",
    "    return 1 if name[-1] in 'aeoiu' else 0\n",
    "\n",
    "test(extract_ends_with_vowel)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Oh yes, that looks promising! What we have here is a way to turn a name into a number, and not just a number, but a number the computer can use to understand whether this is name might be a typical boys or girls name."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try to make a function that uses our numerical version of a name to decide whether it's a girls or boys name.\n",
    "\n",
    "If we look at the histogram above, we can say that if the number is less than 0.5, it's most likely a boys name. Or in Python:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 202,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'boy'"
      ]
     },
     "execution_count": 202,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def guess_gender(name):\n",
    "    feature = extract_ends_with_vowel(name)\n",
    "    if feature < 0.5:\n",
    "        return 'boy'\n",
    "    else:\n",
    "        return 'girl'\n",
    "    \n",
    "guess_gender(\"Jelmer\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great. For my name, it works! We have lists of male and female names, how well does it work for those? Let's try it on 1000 names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 203,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "752 out of 1000 was correct\n",
      "715 out of 1000 was correct\n"
     ]
    }
   ],
   "source": [
    "correct = sum(guess_gender(name) == 'boy' for name in male_names[:1000])\n",
    "print(f'{correct} out of 1000 was correct')\n",
    "\n",
    "correct = sum(guess_gender(name) == 'girl' for name in female_names[:1000])\n",
    "print(f'{correct} out of 1000 was correct')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Not bad for a first attempt, better than 50/50. But definitely not great.\n",
    "\n",
    "We might need to combine this test with other tests. But how? We can make `guess_gender(name)` more complex, add more if-statements, but what are the numbers that we need to compare? What combination of numbers decides whether a name is probably a boys or girls name?\n",
    "\n",
    "# Making decisions\n",
    "Here is another way to look at this problem. Say, you look back at that last histogram where boy names were mostly on the left and girl names mostly on the right. What we did for our `guess_gender(name)` function was drawing a line in the middle of that plot and decide that anything left of the line was boys, right girls.\n",
    "\n",
    "That was in one dimension. One number. We can do that for two dimensions, two numbers as well! Let's draw where the names fall on a plot if we use two numbers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 204,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAV6ElEQVR4nO3df2zcd33H8dc7seOsdZbLmhQTGtujo/Oqjh6VyxoR4WsxKOss+GNsMojBd6oWwTbExLKpUyuVbY0GokNoWqUt0tjRbRB+bExRBh3zmrsqiHR24FpK8abC4jq0rlOWi5KiOE7z2R9338vd+ez72nffu4/Pz4cUxd+7r8/vz9fJy29/vp/vfc05JwCAvza1uwAAwMoIagDwHEENAJ4jqAHAcwQ1AHiuK44X3blzpxscHIzjpQGgI506deoV59yuWs/FEtSDg4OampqK46UBoCOZ2cxyzzH1AQCeI6gBwHMENQB4jqAGAM8R1ADgOYIawIaQSqeUSqfaXcaaENQA4LlY1lEDgC/CLjo7k63YzgSZ9hS0BnTUAOA5OmoAHS3snNdjJx2iowYAz9FRA9gQ1mMnHaKjBgDPReqozey0pAuSXpN0xTk3HGdRAIBrVjP1cbdz7pXYKgEA1MTUBwB4LmpQO0nfNLNTZnag1g5mdsDMpsxs6uzZs82rEAA2uKhBvc85d4ekX5X0e2b29uodnHOHnXPDzrnhXbtq3k0GALAGkYLaOffj4t/zkr4m6a1xFgUAuKZuUJvZ9Wa2LfxY0rskPRt3YQCAgiirPl4n6WtmFu7/Befc47FWBQAoqRvUzrkfSbq9BbUAAGpgeR4AeI6gBgDPEdQA4DmCGgA8R1ADgOcIagDwHEENAJ4jqAHAcwQ1AHiOoAYAzxHUAOA5ghoAPEdQA4DnCGoA8BxBDQBrkEqnlEqnWvK1CGoAaFDcoR3lDi8AgKIwkLMz2dJ2bi6nZF8ytq9JRw0ADcjN5XR+4byyM9nYOmuCGgBWIRNklAkyGhkY0fae7bF20iGmPgBgjZJ9SWWCTKmLzgSZWL4OQQ0AaxBXKNdCUANAg+IObeaoAaABE0GgiSCI9WsQ1ADgOaY+AGANwi56fnKyYns0nW7616KjBoAIWnnJeDU6agBYg7BzjrOTDhHUALCCWpeMS0tXesS5lpqpDwBowGg6HWs3La2iozazzZKmJP3YOTcWX0kA4I+wQ67XSdfruBuxmo76Y5J+0LSvDACIJFJHbWY3Sfo1SYckfTzWigDAQ8t1yPU67maI2lF/VtIfS7ra9AoAACuq21Gb2ZikeefcKTNLrbDfAUkHJKm/v79pBQLAehDn+31E6ajfJundZnZa0hFJ95jZP1bv5Jw77Jwbds4N79q1q8llAsDGVTeonXN/4py7yTk3KGlc0hPOuQ/EXhkAQBLrqAHAe6u6MtE5l5GUiaUSAEBNdNQA4DmCGgA8R1ADgOcIagDwHEENAJ4jqAHAcwQ1AHiOoAYAzxHUAOA5ghoAPEdQA4DnCGoA8BxBDQCeI6gBwHMENQB4jqAGAM95GdQTQaCJIIjltVPpVOm27gA2rvWUBV4GNQC0Qm4uVxHWvob3qm7FFbewi56fnCxt5+ZyOjaeaPhW7OHBz85kK7bjvMU7AP9UZ0EY1uVZ4Fs+eBXU1XJzuXaXAKADlWfL+YXzOvHCCSU+mdD5hfOSpO0929tVWk3mnGv6iw4PD7upqak1f/5EEOjECyf02Xe9UjpwIwMjkhr/CefbT0oA8ar1fz6VTik3l1sSzOF2aHvPdiX7ksoEmZqvE84CjKbTDddpZqecc8O1nvNujjo8gFeuXlly0ACgEeEcdCbIKNmX1Pae7RoZGFGyL1na3mybV/Wa1fPccfCuo66ePwp/0uXvzzelNgAbQ3WWjAyMKDeXK3XI5ftIUuqxM5KkPx35YcXrVHfbIwMjSj12Rr1berV7dlGS9OKebiX7kg111it11N7NUdc6gADQqHCqIzuTXZIv2Zms9l0dWPI5DxwfkMn0F/fMVrxOSjt18fJFST2SpPylvHJzOT1cdVKyWbwL6mprGXQz540ArE/lTV/YSYfddbmxI3ntuzSgW89eL6kQzpJ06O4ZdW3qUu+WXu3rH1BuLlcM52td90PZm3Xl6hUduntGIwODsY3F26DmZB+AZlnxhGCmsHih3EB+qx48PqhbzvZIWtS+9GntU0KH7j5fce6sd0uvLl6+qJGBkVgzy9ugXota67AlOmtgIwsDdCIINDaX17HxRMXzo+m03vvJhP7gmztL3fFD2Zu1WcufVAznrQ8+Hv+JRKnDghoAVpLsS+pgkF7yeP7+vCamCxfYjQwM6hNPZCRdu+jur1P5Uif9UPZmSVLmgzcV/m7Bb/8dFdRh50wnDSBU/Zv2I/sLS/FCYU6MptN6eJnuuNBtJ3To7hlJhSmPVk7PdkxQcyELgEZV58doOq1RFcJ+9pkpffrbt2v3y4uSFlvaENYNajPbKulJFdahdEn6qnPuobgLawSdNIDQaDqtVDqlsblu5S/lpUuFJXbhGuiww66VG4/sL3Teu2cX1SPphvkrLaz8migd9YKke5xzF82sW9IJM/uGc+5kzLVFwpstAWiVPW8e1rnpae0YGmppQ1g3qF3h0sWLxc3u4p/mX84IADHJBBkpqLyQbuxIXvlLeR3aO6ORgUTFxSqlBnDv05KkT+t2SdL70+nY3it/JZHmqM1ss6RTkn5B0qPOuadq7HNA0gFJ6u/vb2aNK6o+sHTSAOLUjqnVVb3Xh5klJH1N0kedc88ut1+j7563FgQ1gNWqlxutzJWmvdeHcy5vZscl7Ze0bFC3AwENoFNFWfWxS9JiMaR/RtI7JX0q9soAIGb1GjxfGsAoHfXrJX2+OE+9SdKXnXPH4i0LABCKsurjGUlvaUEtAIAavLvDCwCgEkENAJ4jqAHAcwQ1AHiOoAYAzxHUAOA5ghoAPEdQA4DnCGoA8BxBDQCeI6gBwHMENQB4jqAGAM8R1ADgOYIaADznXVBPBEFb7vILAL7yLqgBoNma2QCm0qnSTW9XeqyZVnVz2ziFB3F+crJie7lbs9d7HgCkQlacm57WjqGhSPuv5s7jpfBPram0yLwJ6qjCA5Oby0mSRlfxOYQ6sLGEIb144YLmJyeXzYIo4Rzuk53JlrZTL5xR75ZeZWeejvw6a+FNUIcHrtaBLH/s3PS0JGn3hcVl9weA8pAOrdRZ5+ZySqVTFUEsXQvdsDmUpAeODyixNa/dL/dIWtQDlwYkSSeCpg9DkkdBXc+56WlNBEHFQZcqO+vq0K6eTnlkf1LJviShDmwQO4aGSv//u7dt046hoYr//9VdcnkYV0v2JUsfJ7bmlexLan52sridkNT8TjrkXVBXd9K5uZx2zxa657CblqRXu1+TJB0b3yVJOti6EgGsA+UNW9hJ12vSkn1JpR4rTGccfCgjaWmYb+/ZrlxKyt+fK722dEnHxhOx5dC6WvWxY2hI3du26cY771TXpi51bepSJsjowcygJoJA85OTpXmoiSDQaDqt3FxOCz2m53a9qj/a+7QeTp2O9ewsAP8sF9KZIKNMkNHIwIhGBkaUCTLq3dK7ZL/yTjvZl6zorsPHJMWWLd511NK1TvrYeELZmaf1wKUBJbYmdDh1Wg+qML/0kxvXVnpuLrfkIAPoXFGnOseO5DWRCUq/wYdTp5l0Rql0qpQd4fRG+Hx4onJsrrvwQkHzag9511GHB6Se3bOL2j27WLE2Muy2y3/l2T27qJ4Fp8TWhB7K3lxxoAFAKnTWtRq48hOM5xfOl7ZryV/KK38pH8uaaq866okg0NhcvvQT7cHMoMbm8joWJJQJMjooaSITrOm1E3MLyieuKDuTVSqd0tiRPCcWAZRUL0KQpBvmr2jsSF7ZvYXt8jAfTaeLS/QW1LulV4f2zkiSRjTY9Nq8CepUOqWxucJPpN26XtLSM7DlqzqqrzIKz+yG+4UH/ZH9yUJI9/Xo0N5pAUBUvVt6tadvSCMDy6/q6N3Sq2RfcsV9GuVNUEu6Nid9vDAnfWz8Wie9FhNBoL4zi9rkNum62UV9Wrfrhvkr2vPmQc3PTmp+dvkF8AA6R9QLUa6t4lBpKfD85KQ+9IwVzosFla+Xnckqu1caGUjEev7Lm6AOD2AqnSqtUTwYpCVFWw9dHbhhh7696zVdv7hZknTx8kXdoK1xDwVAhwk763JjR/Lad2lAh+4uTHnEef7Lm6Aud2w8UQrp1QgvihlNpzV2JC9JpZBe6DH1bunVb3+nMJ1CJw10vlqXfUtLO+vqZvDGO+8sdda1lvZlgowmMoXVaSMDg7EvUKi76sPM9pjZcTN7zsy+b2Yfi7OgcF1judF0Wg+nTuvFPd3LroceTacrLg1N9iU1kL/WPYfzSADQiPA6jfnJSe2eXSws64v5rZmjdNRXJP2hc+47ZrZN0ikz+w/n3HOxVrYKy73z3o6hIc0+M6Wf3Nilg4+frPgcOmmg85VPqZZvV1vpvYbqaUUDWDeonXMvSXqp+PEFM/uBpDdIamlQZ4KMFBQO+IhW/lWj/GRAj6SB/NaKlSAAsFaNhPparWqO2swGJb1F0lM1njsg6YAk9ff3N6G06Gqtf5SuddhR34cWQGeKOofsazNnzrloO5r1SspKOuSc+5eV9h0eHnZTU1NNKG91lnv3PF8PPgCEzOyUc2641nOROmoz65b0z5L+qV5ItxOBDKAT1Q1qMzNJfyfpB865z8RfUvMQ3AA6QZQ3ZXqbpN+SdI+Z5Yp/7o25LgBAUZRVHyckWQtqAQDU4N3bnAIAKhHUAOA5ghoAPEdQA4DnCGoA8BxBDQCeI6gBwHMENQB4jqAGAM8R1ADgOYIaADxHUAOA5whqAPAcQQ0AniOoAcBzBDUAeG7DBfVEECy5WzkA+CzSzW3Xg6h3HM/N5Qr7xVwPADSLV0GdSqckSZkg09TXnQgCnZue1o6hIe2eXSw9JnEDXAD+8yqo1yIM3PnJSUnSV+66S5L0GydPVux38fJFzczltLu4TWcNYL3wIqjDTjo7k63YbrSzLg/xnuJjr3a/ppnEJZ0YH5QkHWzoKwBA/LwI6kaEUxdhJ7144YKkyumOULIvqdn5KSW2Jpo+vQIAcfEiqMPQbPYc9Y6hIY2m0xXz0eHXoJMGsF54EdTNEM5Jr3SScCII9KAGOYEIYF3xah11Jsgs201HXf98bnpa56anKx4Lg7n6cQCQ/L++wqugXk4431zr8eqDu2NoqGJeuvzzFy9c0PzkpPffFAAo5/3UR62QlZZObVQv0ysP4vDzy7erwxzAxrNcbvg2Pep1UP/9Hbep6/JVbXZWeizsrCeCoOLgrhS+O4aGSvt2b9tWOskIAOuB10EtSQtdTtctFoLaNm8uhXGu7OIVqfYKj3LlYU5IA5Cu5YSvnXTIy6B+ZH9SFy9f1C0LPZI26dXu13Td4iYtdF0tHciH0ymNHckr2ZeM/LqENID1qG5Qm9nnJI1JmnfO3RZ/SUttvbJJJlPPgtMj+wvBnN37tPZdGihdCh4G9nJBTEADWI7v+RBl1Uda0v6Y66hw8PGcPvHE83pxT7d+2n1VL9+0ZcX9d88uspoDQMeq21E75540s8H4S6ntzM8tKjP+Oj2YKZTw/uJPvmPplBJbC1Mf87OT7SoPAGLXtDlqMzsg6YAk9ff3N+U1Dz6eK308kQmWPH9sPKGDwfInEAGgEzQtqJ1zhyUdlqTh4WHXrNcNVYcwb6oEYKPwctXHatFJA+hk6+IScgDYyOoGtZl9UdK3Jf2imZ0xs/viLwsAEIqy6uN9rSgEAFAbUx8A4DmCGgA8R1ADgOcIagDwHEENAJ4jqAHAcwQ1AHiOoAYAzxHUAOA5ghoAPEdQA4DnCGoA8BxBDQCeI6gBwHMENQB4jqAGAM8R1ADgOYIaADxHUAOA5whqAPAcQQ0AniOoAcBzBDUAeI6gBgDPEdQA4DmCGgA8tyGDeiIINBEE7S4DACLZkEENAOtJV7sLaKWwi56fnKzYHk2n21MQAEQQqaM2s/1m9t9m9ryZ3R93UQCAa+p21Ga2WdKjkt4p6YykSTM76px7Lu7imi3snOmkAawnUTrqt0p63jn3I+fcZUlHJL0n3rIAAKEoc9RvkDRbtn1G0q9U72RmByQdkKT+/v6mFBcXOmkA60nTVn045w4754adc8O7du1q1ssCwIYXJah/LGlP2fZNxccAAC0QJagnJb3JzH7ezLZIGpd0NN6yAAChunPUzrkrZvb7kv5d0mZJn3POfT/2ygAAkiJe8OKc+7qkr8dcCwCgBi4hBwDPmXOu+S9qdlbSzBo+daekV5pcjk86eXydPDaps8fXyWOT1s/4BpxzNZfMxRLUa2VmU8654XbXEZdOHl8nj03q7PF18tikzhgfUx8A4DmCGgA851tQH253ATHr5PF18tikzh5fJ49N6oDxeTVHDQBYyreOGgBQhaAGAM+1Jajr3THGzHrM7EvF558ys8HWV7l2Ecb3cTN7zsyeMbP/NLOBdtS5FlHv9mNmv25mzszW1bKoKOMzs98sfv++b2ZfaHWNaxXh32W/mR03s+8W/23e244618LMPmdm82b27DLPm5n9VXHsz5jZHa2usSHOuZb+UeH9Qn4o6Y2Stkh6WtKtVfv8rqS/KX48LulLra4z5vHdLem64scfWS/jizK24n7bJD0p6aSk4XbX3eTv3ZskfVfSjuL2je2uu4ljOyzpI8WPb5V0ut11r2J8b5d0h6Rnl3n+XknfkGSS7pL0VLtrXs2fdnTUUe4Y8x5Jny9+/FVJ7zAza2GNjag7PufccefcT4ubJ1V469j1IOrdfv5c0qckXWplcU0QZXy/I+lR59w5SXLOzbe4xrWKMjYn6WeLH2+X9GIL62uIc+5JSf+3wi7vkfSYKzgpKWFmr29NdY1rR1DXumPMG5bbxzl3RdJ5STe0pLrGRRlfuftU+Em/HtQdW/FXyj3OuX9rZWFNEuV7d4ukW8zsW2Z20sz2t6y6xkQZ2yckfcDMzqjwJmwfbU1pLbHa/5deifTueYiHmX1A0rCkkXbX0gxmtknSZyQFbS4lTl0qTH+kVPhN6Ekz+2XnXL6tVTXH+ySlnXN/aWZ7Jf2Dmd3mnLva7sI2unZ01FHuGFPax8y6VPg17Cctqa5xke6IY2ajkh6Q9G7n3EKLamtUvbFtk3SbpIyZnVZhLvDoOjqhGOV7d0bSUefconPufyX9jwrB7bsoY7tP0pclyTn3bUlbVXhDo06wru9U1Y6gjnLHmKOSPlT8+L2SnnDFMwLrQN3xmdlbJP2tCiG9XuY4pTpjc86dd87tdM4NOucGVZh/f7dzbqo95a5alH+b/6pCNy0z26nCVMiPWlnkGkUZ2wuS3iFJZvZLKgT12ZZWGZ+jkj5YXP1xl6TzzrmX2l1UZG06Q3uvCp3IDyU9UHzsz1T4Ty0V/oF8RdLzkv5L0hvbfda1yeObkPSypFzxz9F219yssVXtm9E6WvUR8XtnKkzvPCfpe5LG211zE8d2q6RvqbAiJCfpXe2ueRVj+6KklyQtqvBbz32SPizpw2Xft0eLY//eevt3ySXkAOA5rkwEAM8R1ADgOYIaADxHUAOA5whqAPAcQQ0AniOoAcBz/w9Wsygu0+CzswAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def test2(feature_extractor_1, feature_extractor_2):\n",
    "    # Because the image will get crowded if I draw all names, I only\n",
    "    # pick 100 names for each group at random.\n",
    "    female_names_sample = random.sample(female_names, 50)\n",
    "    male_names_sample = random.sample(male_names, 50)\n",
    "    \n",
    "    fig, ax = plt.subplots()\n",
    "    ax.scatter(\n",
    "        jitter([feature_extractor_1(name) for name in female_names_sample]),\n",
    "        jitter([feature_extractor_2(name) for name in female_names_sample]),\n",
    "        label='girls',\n",
    "        color='green',\n",
    "        marker='+'\n",
    "    )\n",
    "    ax.scatter(\n",
    "        jitter([feature_extractor_1(name) for name in male_names_sample]),\n",
    "        jitter([feature_extractor_2(name) for name in male_names_sample]),\n",
    "        label='boys',\n",
    "        color='brown',\n",
    "        marker='+'\n",
    "    )\n",
    "    plt.show()\n",
    "\n",
    "    \n",
    "def jitter(numbers):\n",
    "    # This function takes the numbers and nudges them around a bit\n",
    "    # so they are not exactly in the same place. Otherwise we would\n",
    "    # draw all plusses at exactly the same position and you wouldn't\n",
    "    # see how many there were.\n",
    "    stdev = .1 * (max(numbers) - min(numbers))\n",
    "    return [number + random.random() * stdev for number in numbers]\n",
    "\n",
    "\n",
    "test2(extract_ends_with_vowel, extract_number_of_vowels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Okay, I hope you can sort-of draw a line through that plot where you can say that most plusses on one side are green, and on the other side are brown. If you can, you have just made the prototype for a better version of `guess_gender(name)`!\n",
    "\n",
    "# Machine learning\n",
    "*But I thought this was about machine learning!* I hear you say. So far all we have done is express names into numbers and then try to guess the gender based on that number ourselves. We can draw that line, but a computer can also do that for us, right?\n",
    "\n",
    "Well, yes, yes it can! Because the thing we've been doing here is exactly what a *support vector machine* (or SVM) does.\n",
    "\n",
    "## Train & test sets\n",
    "But before we let the computer decide our line, we need to think about how we are going to test whether the computer did any good.\n",
    "\n",
    "We have our lists with names, we can give those to the SVM to learn where to draw the line. We can also use those lists to test whether it can correctly predict the gender. But we don't want to make it too easy for the SVM, we don't want it to just predict names it has already seen. \n",
    "\n",
    "That's why we will now first split our two lists in train and test sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 205,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_male_names = male_names[0:-500] # Take all but the last 500 names\n",
    "test_male_names = male_names[-500:] # Take only the last 500 names\n",
    "\n",
    "train_female_names = female_names[0:-500]\n",
    "test_female_names = female_names[-500:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we set up the support vector machine:\n",
    "1. We derive numbers for all names and make sure we label them correctly as boy or girl name.\n",
    "2. We \"train\" the SVM (the `clf.fit()` call) in which the SVM will try to draw the perfect line that separates the two genders\n",
    "3. We test how well that worked."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 206,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Note this code is a bit more generic because we will be using it later on as well. For now, assume that 'clf' is the SVM.\n",
    "\n",
    "def test_model(clf, extract_features):\n",
    "    # clf is short for classifier, because this SVM will classify the gender\n",
    "\n",
    "    # Features will be our list with our numerical descriptions of names. Each row in the list\n",
    "    # will be a second list of numbers. For now, let's stick to the two numbers we also used\n",
    "    # in the 2d plot.\n",
    "    features = []\n",
    "\n",
    "    # For each row in features we will also have a label, telling whether that row is an\n",
    "    # example of a boys or a girls name. Because we have to use numbers, let's say a 0 is\n",
    "    # a boys name and a 1 is a girls name.\n",
    "    LABEL_MALE = 0\n",
    "    LABEL_FEMALE = 1\n",
    "    labels = []\n",
    "\n",
    "    for name in train_male_names:\n",
    "        features.append(extract_features(name))\n",
    "        labels.append(LABEL_MALE)\n",
    "    \n",
    "    for name in train_female_names:\n",
    "        features.append(extract_features(name))\n",
    "        labels.append(LABEL_FEMALE)\n",
    "    \n",
    "    # Now we have our features and labels, this will tell the SVM to fit the line.\n",
    "    clf.fit(features, labels)\n",
    "    \n",
    "    # How well did it do?\n",
    "    correct = sum(clf.predict([extract_features(name)]) == [LABEL_MALE] for name in test_male_names)\n",
    "    print(f'{correct} out of {len(test_male_names)} male names was correct')\n",
    "\n",
    "    correct = sum(clf.predict([extract_features(name)]) == [LABEL_FEMALE] for name in test_female_names)\n",
    "    print(f'{correct} out of {len(test_female_names)} female names was correct')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And we need to make our 'feature extractor', our function that turns a name into a set of numbers. We can just reuse the extract functions we made earlier, and combine them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 207,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[362] out of 500 male names was correct\n",
      "[354] out of 500 female names was correct\n"
     ]
    }
   ],
   "source": [
    "from sklearn import svm\n",
    "\n",
    "def extract_two_features(name):\n",
    "    return [\n",
    "        extract_number_of_vowels(name),\n",
    "        extract_ends_with_vowel(name)\n",
    "    ]\n",
    "\n",
    "test_model(svm.SVC(), extract_two_features)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Okay, well, euh, it's something. Not entirely great yet. Maybe we need more numbers!\n",
    "\n",
    "Wait, can you just do that? Yes! First we were drawing a line in 1D, then 2D, and we can go to 3D, 4D, *Whatever*D!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 208,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[370] out of 500 male names was correct\n",
      "[351] out of 500 female names was correct\n"
     ]
    }
   ],
   "source": [
    "def extract_features(name):\n",
    "    return [\n",
    "        extract_number_of_vowels(name),\n",
    "        extract_ends_with_vowel(name),\n",
    "        extract_e(name),\n",
    "        extract_length(name),\n",
    "    ]\n",
    "\n",
    "test_model(svm.SVC(), extract_features)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Still not really good. We need a better descriptor.\n",
    "\n",
    "How about just counting how often each letter occurs in a name?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 209,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[254] out of 500 male names was correct\n",
      "[447] out of 500 female names was correct\n"
     ]
    }
   ],
   "source": [
    "import string\n",
    "\n",
    "def extract_features(name):\n",
    "    descriptor = []\n",
    "    for letter in string.ascii_lowercase: # for 'a' to 'z'\n",
    "        descriptor.append(name.lower().count(letter))\n",
    "    return descriptor\n",
    "\n",
    "test_model(svm.SVC(), extract_features)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Please go ahead and try some more variations. For example, you can:\n",
    "- add one for whether the last letter is a specific letter,\n",
    "- or whether the second-to-last letter is a specific letter.\n",
    "- Or you can combine numbers, like the number of vowels in the name divided by the length.\n",
    "\n",
    "## Be consistent\n",
    "One thing to remember though: the numbers have to be consistent. `extract_features(name)` has to return the same amount of numbers for each name. And, each specific number has to have the same meaning! So if the 5th number returned by `extract_features(\"Jelmer\")`, which is 2, tells us how often the letter _e_ occurs in _Jelmer_, the 5th number returned by `extract_features(\"Lucas\")` also has to describe the amount of _e_.\n",
    "\n",
    "Another way to think about it: Going back to the 2D plot we had earlier, we gave each name a position on it by deriving two numbers and using those as _x_ and _y_. It would be very confusing if for one name that _x_ position ment whether the last letter is a vowel, and for another name whether it had an _e_. If your feature extractor translates a name into two numbers, you can imagine it gives the name a place in a two-dimensional room. If it returned three numbers, it would be a three-dimensional room. You might be able to imagine what that looks like. Four dimensions, [maybe you can comprehend](https://en.wikipedia.org/wiki/Hypercube). The last `extract_features()` above gives you 26 numbers, so really we're playing in a 26-dimensional room now! Don't even try to imagine what that looks like. (In machine learning we call this n-dimensional room the feature space.)\n",
    "\n",
    "That reminds me of yet another trap: what is the meaning of the number itself? In the examples above we have used 0 and 1 for whether a vowel is the last letter. We also used the number to tell how often a letter occurs. Wouldn't it make sense to use a number to tell which letter occurs last? It is easy enough to use 1 for _a_, 2 for _b_, 26 for _z_ etc. But if you go back to the n-dimensional room, it would mean that 25 (_y_) and 26 (_z_) are much closer to each other than for example 1 (_a_) and 15 (_o_). That's odd, and a bit meaningless. That is why we should not do that. (If you want to know more, it's the difference between nominal, ordinal, interval and ratio data. The machine learning methods we use here work best when the numbers express a ratio.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# About that data\n",
    "Maybe an SVM is not the best way to make a decision on the typical gender of a name. In our exploration we already concluded that there are quite a few names that are both typically male and female names. You can't draw a boundary line through those points!\n",
    "\n",
    "Now maybe, you say, there are many people that have such a name, but most of them are male. True, I say, but given the data we have, we don't know that. So we can't learn that to our computer…\n",
    "\n",
    "Maybe that's unfair, let's remove those shared names from our dataset. (And because I'm not in the mood for complex filtering for-loop, we'll use a set!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 210,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2443, 4501)"
      ]
     },
     "execution_count": 210,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# \"A ^ B\" means intersection, i.e. keep everything that is in A and also in B at the same time\n",
    "names_in_common = set(male_names) ^ set(female_names)\n",
    "\n",
    "# \"A - B\" with sets means subset, so keep everything in A that is not in B\n",
    "only_male_names = list(set(male_names) - names_in_common)\n",
    "only_female_names = list(set(female_names) - names_in_common)\n",
    "\n",
    "# Split the names into two sets, one for training and one for testing\n",
    "train_male_names = male_names[0:-500] # Take all but the last 500 names\n",
    "test_male_names = male_names[-500:] # Take only the last 500 names\n",
    "\n",
    "train_female_names = female_names[0:-500]\n",
    "test_female_names = female_names[-500:]\n",
    "\n",
    "# To check whether we still have enough names left\n",
    "len(train_male_names), len(train_female_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 211,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[254] out of 500 male names was correct\n",
      "[447] out of 500 female names was correct\n"
     ]
    }
   ],
   "source": [
    "test_model(svm.SVC(), extract_features)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# About the algorithm\n",
    "As I said, maybe an SVM with its single line isn't flexible enough. When we began we began by writing if-rules ourselves. Maybe we can do something like that, but let the computer decide on those rules by itself?\n",
    "\n",
    "Turns out, not a novel idea. That's a decision tree classifier! Let's try it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 212,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[278] out of 500 male names was correct\n",
      "[363] out of 500 female names was correct\n"
     ]
    }
   ],
   "source": [
    "from sklearn import tree\n",
    "test_model(tree.DecisionTreeClassifier(), extract_features)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Not much better. How about using probability theory? Given each of the number, what's the chance the gender is male or female?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[206] out of 500 male names was correct\n",
      "[413] out of 500 female names was correct\n"
     ]
    }
   ],
   "source": [
    "from sklearn import naive_bayes\n",
    "test_model(naive_bayes.GaussianNB(), extract_features)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# About the representation\n",
    "Another thing that might be difficult for the computer is that our numbers are in an odd range. What if we normalize them, so they are always somewhere between 0 and 1?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 214,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[254] out of 500 male names was correct\n",
      "[452] out of 500 female names was correct\n"
     ]
    }
   ],
   "source": [
    "# First, make a list of numbers that is as long as our numerical description of a name\n",
    "ranges = [0] * len(extract_features(''))\n",
    "\n",
    "# Then, determine the maximum each of the numbers can be\n",
    "for name in male_names + female_names:\n",
    "    for n, number in enumerate(extract_features(name)):\n",
    "        ranges[n] = max(number, ranges[n])\n",
    "\n",
    "# Now then, we can make a new extract_features function that uses the original one,\n",
    "# but also uses the maximum numbers we found to make all numbers fall between 0 and 1.\n",
    "def extract_features_normalized(name):\n",
    "    # Use our original name-to-numbers function\n",
    "    numbers = extract_features(name)\n",
    "    \n",
    "    # Normalize them, bringing them from 0-N into 0-1 range\n",
    "    for n, number in enumerate(numbers):\n",
    "        numbers[n] = number / ranges[n]\n",
    "    \n",
    "    return numbers\n",
    "\n",
    "# Let's test it!\n",
    "test_model(svm.SVC(), extract_features_normalized)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Credits\n",
    "This workshop is based on [chapter 6 \"Learning to Classify Text\" of the NLTK book](http://www.nltk.org/book/ch06.html). You should definitely take a look at it if you want to do more with this."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example feature extractors\n",
    "I've written these intentionally at the end. This is my attempt at an elaborate but quite well performing feature extractor for names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 217,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[345] out of 500 male names was correct\n",
      "[434] out of 500 female names was correct\n"
     ]
    }
   ],
   "source": [
    "def extract_features(name):\n",
    "    descriptor = []\n",
    "    \n",
    "    # Does a letter occur in the name\n",
    "    for letter in string.ascii_lowercase: # for 'a' to 'z'\n",
    "        descriptor.append(letter in name.lower())\n",
    "    \n",
    "    # How often does it occur, relatively\n",
    "    for letter in string.ascii_lowercase: # for 'a' to 'z'\n",
    "        descriptor.append(name.lower().count(letter) / len(name))\n",
    "    \n",
    "    # Is it the first letter\n",
    "    for letter in string.ascii_lowercase: # for 'a' to 'z'\n",
    "        descriptor.append(1 if name.lower()[0] == letter else 0)\n",
    "\n",
    "    # Is it the last letter\n",
    "    for letter in string.ascii_lowercase: # for 'a' to 'z'\n",
    "        descriptor.append(1 if name.lower()[-1] == letter else 0)\n",
    "\n",
    "    # Is it the second to last letter\n",
    "    for letter in string.ascii_lowercase: # for 'a' to 'z'\n",
    "        descriptor.append(1 if name.lower()[-2] == letter else 0)\n",
    "        \n",
    "    return descriptor\n",
    "\n",
    "test_model(svm.SVC(), extract_features)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}