answers/17_regular_expressions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regular Expressions\n",
    "\n",
    "In the error handling session we tried to interpret strings as valid heights and weights. This involved looking for text such as \"meter\" or \"kilogram\" in the string, and then extracting the number. This process is called pattern matching, and is best undertaken using a regular expression.\n",
    "\n",
    "Regular expressions have a long history and are available in most programming languages. Python implements a standards-compliant regular expression module, which is called `re`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's create a string that contains a height and see if we can use a regular expression to match that..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "h = \"2 meters\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To search for string \"meters\" in a string, using `re.search`, e.g."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "String contains 'meters'\n"
     ]
    }
   ],
   "source": [
    "if re.search(\"meters\", h):\n",
    "    print(\"String contains 'meters'\")\n",
    "else:\n",
    "    print(\"No match\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`re.search` returns a match object if there is a match, or `None` if there isn't."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = re.search(\"meters\", h)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<_sre.SRE_Match object; span=(2, 8), match='meters'>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This matches \"meters\", but what about \"meter\". \"meter\" is \"meters\" without an \"s\". You can specify that a letter is matched 0 or 1 times using \"?\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "h = \"2 meter\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = re.search(\"meters?\", h)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<_sre.SRE_Match object; span=(2, 7), match='meter'>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, this has still not worked, as we match \"meters\" in the middle of the string. We need to match \"meters\" only at the end of the string. We do this using \"$\", which means match at end of string"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = re.search(\"meters?$\", h)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<_sre.SRE_Match object; span=(2, 7), match='meter'>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also want to be able to match \"m\" as well as \"meters\". To do this, we need to use the \"or\" operator, which is \"|\". It is a good idea to put this in round brackets to make both sides of the \"or\" statement clear."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "h = \"2 m\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = re.search(\"(m|meters?)$\", h)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<_sre.SRE_Match object; span=(2, 3), match='m'>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we want to match the number, e.g. \"X meters\", where \"X\" is a number. You can use \"\\d\" to represent any number. For example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "h = \"2 meters\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = re.search(\"\\d (m|meters?)$\", h)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<_sre.SRE_Match object; span=(0, 8), match='2 meters'>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A problem with the above example is that it only matches a number with a single digit, as \"\\d\" only matches a single number. To match one or more digits, we need to put a \"+\" afterwards, as this means \"match one or more\", e.g."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "h = \"10 meters\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = re.search(\"\\d+ (m|meters?)$\", h)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<_sre.SRE_Match object; span=(0, 9), match='10 meters'>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This match breaks if the number is has decimal point, as it doesn't match the \"\\d\". To match a decimal point, you need to use \"\\\\.\", and also \"?\", which means \"match 0 or 1 decimal points\", and then \"\\d*\", which means \"match 0 or more digits\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "h = \"1.5 meters\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = re.search(\"\\d+\\.?\\d* (m|meters?)$\", h)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<_sre.SRE_Match object; span=(0, 10), match='1.5 meters'>"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The number must match at the beginning of the string. We use \"^\" to mean match at start..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "h = \"some 1.8 meters\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = re.search(\"^\\d+\\.?\\d* (m|meters?)$\", h)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we want this match to be case insensitive, and would like the user to be free to use as many spaces as they want between the number and the unit, before the string or after the string... To do this we use \"\\s*\" to represent any number of spaces, and match using `re.IGNORECASE`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "h = \"   1.8 METers   \""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = re.search(\"^\\s*\\d+\\.?\\d*\\s*(m|meters?)\\s*$\", h, re.IGNORECASE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<_sre.SRE_Match object; span=(0, 16), match='   1.8 METers   '>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The round brackets do more than just groups parts of your search. They also allow you extract the parts that match."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('METers',)"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.groups()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can place round brackets around the parts of the match you want to capture. In this case, we want to get the number..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = re.search(\"^\\s*(\\d+\\.?\\d*)\\s*(m|meters?)\\s*$\", h, re.IGNORECASE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('1.8', 'METers')"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.groups()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As `m.groups()[0]` contains the match of the first set of round brackets (which is the number), then we can get the number using `m.groups()[0]`. This enables us to rewrite the `string_to_height` function from the last section as;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "def string_to_height(height):\n",
    "    \"\"\"Parse the passed string as a height. Valid formats are 'X m', 'X meters' etc.\"\"\" \n",
    "    m = re.search(\"^\\s*(\\d+\\.?\\d*)\\s*(m|meters?)\\s*$\", height, re.IGNORECASE)\n",
    "    \n",
    "    if m:\n",
    "        return float(m.groups()[0])\n",
    "    else:\n",
    "        raise TypeError(\"Cannot extract a valid height from '%s'\" % height)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "h = string_to_height(\"   1.5    meters   \")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.5"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "h"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise\n",
    "\n",
    "## Exercise 1\n",
    "\n",
    "Rewrite your `string_to_weight` function using regular expressions. Check that it responds correctly to a range of valid and invalid weights."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "def string_to_weight(weight):\n",
    "    \"\"\"Parse the passed string as a weight. Valid formats are 'X kg', 'X kilos', 'X kilograms' etc.\"\"\" \n",
    "    m = re.search(\"^\\s*(\\d+\\.?\\d*)\\s*(kgs?|kilos?|kilograms?)\\s*$\", weight, re.IGNORECASE)\n",
    "    \n",
    "    if m:\n",
    "        return float(m.groups()[0])\n",
    "    else:\n",
    "        raise TypeError(\"Cannot extract a valid weight from '%s'\" % weight)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(23.5, 5.0, 10.0)"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "string_to_weight(\"23.5 kilos\"), string_to_weight(\"5kg\"), string_to_weight(\"10 kilogram\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2\n",
    "\n",
    "Update string_to_height so that it can also understand heights in both meters and centimeters (returning the height in meters), and update string_to_weight so that it can also understand weights in both grams and kilograms (returning the weight in kilograms). Note that you may find it easier to separate the number from the units. You can do this using the below function to divide the string into the number and units. This uses \"\\w\" to match any word character."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_number_and_unit(s):\n",
    "    \"\"\"Interpret the passed string 's' as \"X units\", where \"X\" is a number and\n",
    "       \"unit\" is the unit. Returns the number and (lowercased) unit\n",
    "    \"\"\"\n",
    "    m = re.search(\"^\\s*(\\d+\\.?\\d*)\\s*(\\w+)\\s*$\", s, re.IGNORECASE)\n",
    "\n",
    "    if m:\n",
    "        number = float(m.groups()[0])\n",
    "        unit = m.groups()[1].lower()\n",
    "        return (number, unit)\n",
    "    else:\n",
    "        raise TypeError(\"Cannot extract a valid 'number unit' from '%s'\" % s)       "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "def string_to_height(height):\n",
    "    \"\"\"Parse the passed string as a height. Valid formats are 'X m', 'X centimeters' etc.\"\"\" \n",
    "    (number, unit) = get_number_and_unit(height)\n",
    "    \n",
    "    if re.search(\"cm|centimeters?\", unit):\n",
    "        return number / 100.0\n",
    "\n",
    "    elif re.search(\"m|meters?\", unit):\n",
    "        return number\n",
    "    else:\n",
    "        raise TypeError(\"Cannot convert a number with units '%s' to a valid height\" % unit)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "def string_to_weight(weight):\n",
    "    \"\"\"Parse the passed string as a weight. Valid formats are 'X kg', 'X grams' etc.\"\"\" \n",
    "    (number, unit) = get_number_and_unit(weight)\n",
    "    \n",
    "    if re.search(\"kgs?|kilos?|kilograms?\", unit):\n",
    "        return number\n",
    "\n",
    "    elif re.search(\"g|grams?\", unit):\n",
    "        return number / 1000.0\n",
    "    else:\n",
    "        raise TypeError(\"Cannot convert a number with units '%s' to a valid weight\" % unit)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.55, 2.0, 15.0)"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "string_to_height(\"55 cm\"), string_to_height(\"2m\"), string_to_height(\"15meters\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.015, 5.0, 0.005)"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "string_to_weight(\"15g\"), string_to_weight(\"5 kilograms\"), string_to_weight(\"5gram\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}