notebooks/2_Text.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PgEPTsqKbpRd"
   },
   "source": [
    "# Python basics 2: Text\n",
    "\n",
    "This notebook contains more basics of Python. Use it as a reference whenever needed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bPXO4Q-5bpRf"
   },
   "source": [
    "## Python Syntax"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TwSFTEKlbpRg"
   },
   "source": [
    "### The Significant Whitespace\n",
    "\n",
    "Most program languages use characters (e.g. `{...}`) or keywords (e.g. `begin ... end`) to delimitate blocks of codes. But, when writing Python code, you rely on **indentation** to structure your programs. \n",
    "\n",
    "All programming languages allow you to indent (and you should!), but in Python you **have to.** Otherwise, you'll receive an IndentationError and your code won't work!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "P841qNdsbpRg"
   },
   "source": [
    "#### How Indentation Works\n",
    "\n",
    "- All statements with the same distance from the left border belong of the same block of code. This is related to the _scope_ of your code. \n",
    "- Sub-blocks are more indented, while the block ends at the line less indented.\n",
    "- It is recommended to use **4 spaces** per indentation level. However, a tab character (`\\t`) can also be used. Often, you have a setting in your favorite editor that automatically translates tabs to 4 spaces. \n",
    "- When a statement is too long, it's good practice to avoid lines of code longer than 79 characters, it can be split with `\"\\\"` at the end of the line.\n",
    "- **Never mix** spaces and tabs in a single source file. This will raise an error when you try to execute the code, but is also very hard to spot by hand. To help you, you can set your text editor to display whitespace characters.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BIWCuPkqbpRg"
   },
   "source": [
    "#### Recommended Reading on Python syntax and style\n",
    "[PEP 8 - Style Guide for Python Code](https://www.python.org/dev/peps/pep-0008/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "t69K9sxSbpRh"
   },
   "source": [
    "##### The code is way more readable:\n",
    "\n",
    "\n",
    "```python\n",
    "# input() reads from standard input (e.g. keyboard)\n",
    "n_string = input('Enter a number, please:')\n",
    "\n",
    "if not n_string.isdigit():\n",
    "    print(\"This isn't a number...\")\n",
    "else:\n",
    "    n = int(n_string)\n",
    "    if n == 0:\n",
    "        print(\"Zero? Why zero?\")\n",
    "    elif n % 2 == 0:\n",
    "        print(\"Even\")\n",
    "    else:\n",
    "        print(\"Odd\")\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aBD18s9ZmAye"
   },
   "source": [
    "### Strings\n",
    "\n",
    "Strings are **immutable**, sequences of characters (so, they are **homogeneous**)\n",
    "\n",
    "In the Python language, strings are defined by single `'` or double quotes `\"`,  and their elements are contiguous."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "6OnqjW8KmAye"
   },
   "outputs": [],
   "source": [
    "demo_string = \"Does Cersei have any friends?\"\n",
    "print(demo_string)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "qbbbfVKamAye"
   },
   "outputs": [],
   "source": [
    "# Using single or double quotes is indifferent\n",
    "demo_string_single = 'Does Cersei have any friends?'\n",
    "demo_string == demo_string_single"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "VSGs9ZMfmAyf"
   },
   "outputs": [],
   "source": [
    "# Remember: Digits are not numbers! (frequent source of debugging frustration)\n",
    "\n",
    "print(type(\"1979\") == type(1979))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "1UgPCLTYmAyf"
   },
   "outputs": [],
   "source": [
    "print('\"1979\" type is: ' + str(type(\"1979\")))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "LmLCmwQTmAyf"
   },
   "outputs": [],
   "source": [
    "print('1979 type is: ' + str(type(1979)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Dx3AMRYBmAyf"
   },
   "source": [
    "In some ways, strings behave the same as lits: by being immutable sequences, strings accept all the sequences methods. However, they not support item assignment as they are immutable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ECO7gPhqmAyf"
   },
   "outputs": [],
   "source": [
    "# You can check the length (in characters, whitespaces count!) of a string\n",
    "len(demo_string)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "FS-6kKHHmAyg"
   },
   "outputs": [],
   "source": [
    "# Strings can be looked for in other strings\n",
    "\"Cersei\" in demo_string"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Yeoqca6FmAyg"
   },
   "outputs": [],
   "source": [
    "# How many 'a's do we have in our string?\n",
    "demo_string.count(\"a\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "J-CqwSOdmAyh"
   },
   "outputs": [],
   "source": [
    "# Concatenation is possible\n",
    "demo_string += \"\\nNoway!\"\n",
    "demo_string"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "l3jC4e6LmAyh"
   },
   "outputs": [],
   "source": [
    "print(demo_string)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aijPKDOPmAyh"
   },
   "source": [
    "The escape sequence `\\n` indicates the end of the line. \n",
    "\n",
    "Python escape sequences are introduced by the escape character `\\`, whose goal is to signal the interpreter that the following character has an \"unusual\" interpretation. Here is a partial list:\n",
    "\n",
    "| Escape Sequence|         Meaning|\n",
    "|:--------------:|:---------------:|\n",
    "|              \\\\\\\\|  backslash|\n",
    "|              \\\\'|  single quote|\n",
    "|              \\\\b|  backspace|\n",
    "|              \\\\n|  new line|\n",
    "|              \\\\t|  horizontal tab|\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "AZIW3j81mAyi"
   },
   "outputs": [],
   "source": [
    "# Nicely print our string!\n",
    "print(demo_string)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Gf5Fpn9fmAyi",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# What does a backspace do?\n",
    "print(demo_string  +\"\\b\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "-9EFZSEWmAyj"
   },
   "outputs": [],
   "source": [
    "# The escape character may be useful when you need single quotes inside a single quote-marked string...\n",
    "'Can any of you pronounce \\'s-Hertogenbosch?'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "a-vwBersmAyk"
   },
   "outputs": [],
   "source": [
    "# but this solution is preferred (when possible)\n",
    "\"Can any of you pronounce 's-Hertogenbosch?\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QDWZTabamAyk"
   },
   "source": [
    "Strings that span multiple lines can be written in a readable form by using the sequence `\"\"\"` as a delimiter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Kem3RR91mAyk"
   },
   "outputs": [],
   "source": [
    "print(\"\"\"Unsealed, on a porch a letter sat\n",
    "Then you said, \"I wanna leave it again\"\n",
    "Once I saw her on a beach of weathered sand\n",
    "And on the sand I wanna leave her again\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2FlXMgW_mAyM"
   },
   "source": [
    "In some ways, string behave the same as lists. For instance, you can check the length of a string, or check if a character sequence is part of a string:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9U8FswPhmAyl"
   },
   "source": [
    "### Quiz"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "i82TbsndmAyl"
   },
   "source": [
    "Run the cell below. Where do all the '\\n's come from?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "OOj9TxtFmAyl"
   },
   "outputs": [],
   "source": [
    "\"\"\"\"On a weekend I wanna wish it all away, yeah\n",
    "And they called and I said that I'll go\n",
    "And I said that I'll call out again\n",
    "And the reason I ought ta leave her calm, I know\n",
    "I said, \"I don't know whether I'm the boxer or the bag\"\"\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xhimsX8-mAym"
   },
   "source": [
    "#### String Methods\n",
    "\n",
    "Strings have a buch of dedicated methods (see the [documentation](https://docs.python.org/3.8/library/stdtypes.html) for a complete list), that allows them to be both inspected or manipulated (they are not modified, rather a **new object** is returned). \n",
    "\n",
    "The following are used the most:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "TWF4n7lbmAym"
   },
   "outputs": [],
   "source": [
    "# Is the string composed solely of 1. digits 2. alphabetic characters 3. both?\n",
    "print('100'.isdigit())\n",
    "print('cat'.isalpha())\n",
    "print('my cat is 100'.isalnum())  # Why False?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "SreTtIeYmAym"
   },
   "outputs": [],
   "source": [
    "print(demo_string)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2RmcIDdcmAym"
   },
   "source": [
    "The `.startswith()` and `.endswith()` methods are useful:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "DcphuqOtmAym"
   },
   "outputs": [],
   "source": [
    "# Does the string starts or ends with a given sequence of characters?\n",
    "print(demo_string.startswith(\"d\"))  # it is case sensitive !!!\n",
    "print(demo_string.endswith(\"Noway!\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0431Dh5tmAyn"
   },
   "source": [
    "If you want to compare strings, or want to have them stored in a normalized way, you can use the `.upper()` and `.lower()` methods:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "mBGyjgofmAyn"
   },
   "outputs": [],
   "source": [
    "# Change case to all the characters of a string\n",
    "print(demo_string_single.upper())\n",
    "print(demo_string_single.lower())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6OcrVbU8mAyn"
   },
   "source": [
    "If you want to get rid of unwanted characters at the beginning or at the end of a string, you can use `.strip()`. This is commonly used to remove all whitespace (i.e. spaces, linebreaks, tabs) from the string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "pXt1IRAtmAyo"
   },
   "outputs": [],
   "source": [
    "# Remove a given character (default is any whitespace) from the beginning and the end of a string\n",
    "\n",
    "text1 = \"Twice minus: - before and after -\"\n",
    "text2 = \"  \\t  Too much space?\"\n",
    "\n",
    "print(\"Before:\")\n",
    "print(text1)\n",
    "print(text2)\n",
    "print()\n",
    "\n",
    "print(\"After:\")\n",
    "print(text1.strip(\"-\"))\n",
    "print(text2.strip())\n",
    "print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hmvBna4zmAyo"
   },
   "source": [
    "You can remove one or multiple elements from a list by using `.replace()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "CEOid6T3mAyo"
   },
   "outputs": [],
   "source": [
    "# Replace a given sequence of characters with another \n",
    "print(demo_string.replace(\"Cersei\", \"Melisandre\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "M0RZKf25mAyo"
   },
   "outputs": [],
   "source": [
    "# Or if you want to completely remove a character or series of characters,\n",
    "# simply replace it by an empty string! You can also chain these. \n",
    "print(demo_string.replace(\"?\", \"\").replace(\"!\", \"\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sC2DXTe-mAyp"
   },
   "source": [
    "#### From strings to lists"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WA3OU9Q9mAyp"
   },
   "source": [
    "A string can be transformed into a list of string by splitting it on a given character. This is done through the `.split()` method. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "6uE3td6imAyp"
   },
   "outputs": [],
   "source": [
    "# By whitespace\n",
    "demo_string.split(\" \")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "krUm1CBimAyp"
   },
   "outputs": [],
   "source": [
    "# However, the default character is any white line (that's convenient!)\n",
    "demo_string.split()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "za1Lkg0HmAyq"
   },
   "outputs": [],
   "source": [
    "# the maximun number of splits can be specified\n",
    "demo_string.split(\" \", 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IXtJoYcAmAyq"
   },
   "source": [
    "#### From lists to strings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rkT8Rn8jmAyq"
   },
   "source": [
    "The inverse operation is possible, a list of strings can be joined by a single character using `join()` on another string. It's argument is the list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "jGoHyN0CmAyq"
   },
   "outputs": [],
   "source": [
    "# A whitespace\n",
    "\" \".join([\"One\",\"Two\",\"Three\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "74BhY6eUmAyq"
   },
   "outputs": [],
   "source": [
    "# An hyphen\n",
    "\"-\".join([\"One\",\"Two\",\"Three\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "TxLEf1TYmAyq"
   },
   "outputs": [],
   "source": [
    "# Any string basically\n",
    "predifined_joinchar = \"🌞\"\n",
    "predifined_joinchar.join([\"Eins\",\"Zwei\",\"Drei\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ysKaOB6amAyr"
   },
   "outputs": [],
   "source": [
    "# Or no characters at all\n",
    "\"\".join([\"Super\", \"cali\", \"fragilistic\", \"expiali\", \"docious\"] )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7CR5wvyimAyr"
   },
   "source": [
    "Keep in mind that you can only join elements of type string. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "prXkMsUjmAyr"
   },
   "outputs": [],
   "source": [
    "# Raises a TypeError\n",
    "\" \".join([\"This\", \"is\", \"notebook\", \"number\", 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hBr4KQzpmAyu"
   },
   "source": [
    "### Dictionaries\n",
    "\n",
    "Dictionaries are **associative arrays** mapping **immutable** types (string, numbers, tuples...) to arbitrary objects of any kind (all datatypes you earlier saw, variables, functions, modules...). Intuitively, they can be thought as collections of objects that we can recall by means of a unique key. \n",
    "\n",
    "To visualize a Python dictionary you can think of a telephone book, in which people names are the unique keys that you use to retrieve difference kinds of information (phone numbers, street address, mail address...). The same telephone number, street address or other information can be present in the entries of more people, but a label cannot be associated with more than one entry. \n",
    "\n",
    "In text processing, dictionaries are often used for counting words, where each word is a key and the number of times it occurs in the text is the value.\n",
    "\n",
    "In Python, dictionaries are defined by curly brackets `{}`, in which key-value pairs are separated by commas and joint by colons."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "lDM2PLycmAyv"
   },
   "outputs": [],
   "source": [
    "# An English-Dutch dictionary of colors\n",
    "\n",
    "color2kleur = {\n",
    "    \"black\": \"zwart\",\n",
    "    \"white\": \"wit\",\n",
    "    \"red\": \"rood\",\n",
    "    \"yellow\": \"geel\"\n",
    "}\n",
    "\n",
    "print(color2kleur)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "GrrV1A-gmAyv"
   },
   "outputs": [],
   "source": [
    "# Values can be recalled by their keys\n",
    "color2kleur[\"white\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "NvGQFsrdmAyv"
   },
   "outputs": [],
   "source": [
    "# We can change a value associated with a key\n",
    "color2kleur[\"white\"] = \"sneeuwwit\"\n",
    "\n",
    "print(color2kleur)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "AdxpH3yImAyv"
   },
   "outputs": [],
   "source": [
    "# If the key is missing a new key: value pair is added\n",
    "color2kleur[\"blue\"] = \"blauw\"\n",
    "\n",
    "print(color2kleur)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "F9eB0loGmAyv"
   },
   "outputs": [],
   "source": [
    "# {Key: value} can be deleted with the command \"del()\"\n",
    "del(color2kleur[\"blue\"])\n",
    "print(color2kleur)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "hXyIkdUSmAyw"
   },
   "outputs": [],
   "source": [
    "# Check if a dictionary has a given key\n",
    "\"blue\" in color2kleur"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "bWVydw5VmAyw"
   },
   "outputs": [],
   "source": [
    "# Count the number of entries in a dictionary\n",
    "len(color2kleur)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YRKzzvjQmAyw"
   },
   "source": [
    "### Quiz\n",
    "\n",
    "What you're actually doing by calling the dict that way, is calling the `.keys()` method. Try using the following methods below on the `color2kleur` dictionary:\n",
    "\n",
    "1. `.keys()` vs. just calling the dictionary by `color2kleur`\n",
    "2. `.values()`\n",
    "3. `.items()`\n",
    "    \n",
    "What is the datatype that is returned for each of these methods? And what is de datatype inside these 'iterables'?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "icDTmUP-mAyw"
   },
   "outputs": [],
   "source": [
    "# You code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FXY9PzofmAyw"
   },
   "source": [
    "#### Iterating over a Dictionary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "5rL_YIXgmAyw"
   },
   "outputs": [],
   "source": [
    "# iterate over dictionary keys:\n",
    "print(list(color2kleur))\n",
    "print(list(color2kleur.keys()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "qQGYGuRvmAyx"
   },
   "outputs": [],
   "source": [
    "# iterate over dictionary values:\n",
    "print(list(color2kleur.values()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "-Vn_EOnnmAyx"
   },
   "outputs": [],
   "source": [
    "# iterate over dictionary key-value pairs:\n",
    "print(list(color2kleur.items()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "r3kPJNMIWjol"
   },
   "source": [
    "## String formatting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IlzCXlG5Wjol"
   },
   "source": [
    "Instead of writing a series of `print()` statements with multiple arguments, or concatenating (by `+`) strings, you can also use a Python string formatting method, called `f-strings`. More information can be read in PEP 498: https://www.python.org/dev/peps/pep-0498/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2WFp8i4pWjom"
   },
   "source": [
    "You can define a string as a template by inserting `{ }` characters with a variable name or expression in between. For this to work, you have to type an `f` in front of the `'`, `\"` or `\"\"\"` start of the string definition. When defined, the string will read with the string value of the variable or the expression filled in. \n",
    "\n",
    "```python\n",
    "name = \"Joe\"\n",
    "text = f\"My name is {name}.\"\n",
    "```\n",
    "\n",
    "Again, if you need a `'` or `\"` in your expression, use the other variant in the Python source code to declare the string. Writing:\n",
    "\n",
    "```python\n",
    "f'This is my {example}.'\n",
    "```\n",
    "\n",
    "is equivalent to:\n",
    "\n",
    "```python\n",
    "f\"This is my {example}.\"\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ZsWlOdzCWjom"
   },
   "outputs": [],
   "source": [
    "name = \"Joe\"\n",
    "text = f\"My name is {name}.\"\n",
    "\n",
    "print(text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "XtKqnCWRWjoo"
   },
   "outputs": [],
   "source": [
    "day = \"Monday\"\n",
    "weather = \"Sunny\"\n",
    "n_messages = 8\n",
    "\n",
    "test_dict = {'test': 'test_value'}\n",
    "\n",
    "text = f\"\"\"\n",
    "Today is {day}. \n",
    "The weather is {weather.lower()} and you have {n_messages} unread messages. \n",
    "The first three letters of the weekday: {day[:3]}\n",
    "An example expression is: {15 ** 2 = }\n",
    "\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "\n",
    "text = f'Test by selecting key: {test_dict[\"test\"]}'\n",
    "\n",
    "\n",
    "print(text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rcz2ZzX8Wjoo"
   },
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6iczszohWjoo"
   },
   "source": [
    "## Regular expressions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "R0KFKUT9Wjoo"
   },
   "source": [
    "Using regular expressions can be very useful when working with texts. It is a powerful search mechanism by which you can search on patterns, instead of 'exact matches'. But, they can be difficult to grasp, at first sight.\n",
    "\n",
    "A **regular expression**, for instance, allows you to substitute all digits in a text, following another text sequence, or to find all urls, phone numbers, or email addresses. Or any text, that meets a particular condition.\n",
    "\n",
    "See the Python manual for the `re` module for more info: https://docs.python.org/3/library/re.html\n",
    "\n",
    "You can/should use a cheatsheet when writing a regular expression. A nice website to write and test them is: https://regex101.com/. \n",
    "\n",
    "Some examples of commonly used expressions:\n",
    "\n",
    "* `\\d` for all digits 0-9\n",
    "* `\\w` for any word character\n",
    "* `[abc]` for a set of characters (here: a, b, c)\n",
    "* `.` any character\n",
    "* `?` the preceding character/pattern 0 or 1 times\n",
    "* `*` the preceding character/pattern 0 or multiple times\n",
    "* `+` the preceding character/pattern 1 or multiple times\n",
    "* `{1,2}` 1 or 2 times\n",
    "* `^` the start of the string\n",
    "* `$` the end of the string\n",
    "* `|` or\n",
    "* `()` capture group (only return this part)\n",
    "\n",
    "In many text editors (e.g. VSCode) there is also an option to search (and replace) with the help of regular expressions. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UHXRd47uWjop"
   },
   "source": [
    "Python has a regex module built in. When working with a regular expression, you have to import it first:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "FySvVyNZWjop"
   },
   "outputs": [],
   "source": [
    "import re"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SVPG0dw8Wjop"
   },
   "source": [
    "You can use a regular expression for **finding** occurences in a text. Let's say we want to filter out all web urls in a text:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "E4ETED8GWjoq"
   },
   "outputs": [],
   "source": [
    "text = \"\"\"\n",
    "There are various search engines on the web. \n",
    "There is https://www.google.com/, but also https://www.bing.com/. \n",
    "A more privacy friendly alternative is https://duckduckgo.com/. \n",
    "And who remembers http://www.altavista.com/?\n",
    "\"\"\"\n",
    "\n",
    "re.findall(r'https?://.+?/', text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "nljkoZR6Wjoq"
   },
   "outputs": [],
   "source": [
    "# Copied from https://www.imdb.com/search/title/?groups=top_250&sort=user_rating\n",
    "\n",
    "text = \"\"\"\n",
    "1. The Shawshank Redemption (1994)\n",
    "12 | 142 min | Drama\n",
    "\n",
    " 9,3  Rate this 80 Metascore\n",
    "Two imprisoned men bond over a number of years, finding solace and eventual redemption through acts of common decency.\n",
    "\n",
    "Director: Frank Darabont | Stars: Tim Robbins, Morgan Freeman, Bob Gunton, William Sadler\n",
    "\n",
    "Votes: 2.355.643 | Gross: $28.34M\n",
    "\n",
    "2. The Godfather (1972)\n",
    "16 | 175 min | Crime, Drama\n",
    "\n",
    " 9,2  Rate this 100 Metascore\n",
    "An organized crime dynasty's aging patriarch transfers control of his clandestine empire to his reluctant son.\n",
    "\n",
    "Director: Francis Ford Coppola | Stars: Marlon Brando, Al Pacino, James Caan, Diane Keaton\n",
    "\n",
    "Votes: 1.630.157 | Gross: $134.97M\n",
    "\n",
    "3. The Dark Knight (2008)\n",
    "16 | 152 min | Action, Crime, Drama\n",
    "\n",
    " 9,0  Rate this 84 Metascore\n",
    "When the menace known as the Joker wreaks havoc and chaos on the people of Gotham, Batman must accept one of the greatest psychological and physical tests of his ability to fight injustice.\n",
    "\n",
    "Director: Christopher Nolan | Stars: Christian Bale, Heath Ledger, Aaron Eckhart, Michael Caine\n",
    "\n",
    "Votes: 2.315.134 | Gross: $534.86M\n",
    "\"\"\"\n",
    "\n",
    "titles = re.findall(r'\\d{1,2}\\. (.+)', text)\n",
    "titles"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ojLPhJ-0Wjor"
   },
   "source": [
    "### Quiz\n",
    "Try to get a list of all directors. And the gross income. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "wvFlrH_BWjor"
   },
   "outputs": [],
   "source": [
    "# All directors\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "0-uZoKO2Wjos"
   },
   "outputs": [],
   "source": [
    "# Gross income\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-x_xCtNQWjos"
   },
   "source": [
    "Or, you can use a regular expression to **replace** a character sequence. This is an equivalent to the `.replace()` function, but allows more variance in the string matching."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "zsNVQetBWjos"
   },
   "outputs": [],
   "source": [
    "text = \"\"\"\n",
    "Tim Robbins\n",
    "Morgan Freeman\n",
    "Bob Gunton\n",
    "William Sadler\n",
    "Marlon Brando\n",
    "Al Pacino\n",
    "James Caan\n",
    "Diane Keaton\n",
    "Christian Bale\n",
    "Heath Ledger\n",
    "Aaron Eckhart\n",
    "Michael Caine\n",
    "\"\"\"\n",
    "\n",
    "# Hint: test this with https://regex101.com/\n",
    "new_text = re.sub(r\"(?:(\\w)\\w+) (\\w+)\", r\"\\1. \\2\", text)\n",
    "\n",
    "print(new_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TBbu5aT9l1BK"
   },
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nb6CksgtbpR1"
   },
   "source": [
    "# Exercises\n",
    "\n",
    "## Reading\n",
    "* Have another look at Python's built-in types: https://docs.python.org/3.8/library/stdtypes.html. Be sure to check out the part on [string methods](https://docs.python.org/3.8/library/stdtypes.html#string-methods) in particular. You can skip the other sections. \n",
    "\n",
    "## Tasks\n",
    "* Try out the quizzes that we did not get to in the lecture!\n",
    "* Try the exercises below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FJIuAyjkbpR6"
   },
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "E5tEL618mAy3",
    "tags": []
   },
   "source": [
    "### Excercise 1\n",
    "\n",
    "Italian nobles tends to have an awful lot of names. For instance, \"Vittorio Emanuele di Savoia\" (or \"Vittorio\" for close friends) has the 12 names listed in `full_name`. \n",
    "\n",
    "Can you find a pythonic way to eliminate the less used names from this string?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "nwffyJv2mAy3"
   },
   "outputs": [],
   "source": [
    "full_name = \"Vittorio Emanuele Alberto Carlo Teodoro Umberto Bonifacio Amedeo Damiano Bernardino Gennaro Maria di Savoia\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "TyqHvGIcmAy3"
   },
   "outputs": [],
   "source": [
    "# Your code here\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9DLqwF1NmAy4"
   },
   "source": [
    "### Exercise 2\n",
    "\n",
    "The code in the next cell creates a variable called `zen_text`, and assigns it a nicely formatted version of the textual elements of the Zen of Python.\n",
    "\n",
    "* Count the number of in this manifesto of:\n",
    "    1. Characters\n",
    "    2. Words\n",
    "    3. Unique words\n",
    "    3. Non-empty lines\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "b1x3rx_LmAy4"
   },
   "outputs": [],
   "source": [
    "import this\n",
    "zen_text = ''.join(this.d.get(el, el) for el in this.s)  # forget this complicated pattern for now"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ZaykSfG_mAy4"
   },
   "outputs": [],
   "source": [
    "# Your code here\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wayILRb0bpR4"
   },
   "source": [
    "### Exercise 3\n",
    "Remember the quiz from the previous notebook with the following code:\n",
    "    \n",
    "```python\n",
    "homogeneous_list = [\"1\", \"56\", \"33\", \"8\", \"220\", \"9\"]\n",
    "homogeneous_list.sort()\n",
    "homogeneous_list\n",
    "```\n",
    "\n",
    "Try to fix this so that it sorts the list as if its elements were integers!\n",
    "\n",
    "Can you then do the same for this list, so that Elephant is after canary?\n",
    "\n",
    "```python\n",
    "homogeneous_list = [\"canary\", \"hippo\", \"kangaroo\", \"narwhal\", \"Elephant\", \"raccoon\", \"yak\", \"ant\"]\n",
    "homogeneous_list.sort()\n",
    "homogeneous_list\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "cLbfTPBJbpR5"
   },
   "outputs": [],
   "source": [
    "# Your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pX5hC93-mAy4"
   },
   "source": [
    "### Exercise 4\n",
    "\n",
    "The dictionary in the following cell reports, for each Scrubs character, a dictionary mapping is given with the name of the actor, the age of the character and its credentials.\n",
    "\n",
    "Write code to answer the following questions:\n",
    "\n",
    "- What are the **names of the actors** of the cast?\n",
    "\n",
    "- What is the **average age** of the characters?\n",
    "\n",
    "- How **many M.D.s** are there in the main cast?\n",
    "\n",
    "- Which character is not listed in the credentials dictionary?\n",
    "\n",
    "Hint: check the dictionary and set methods"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ePZx4vLXmAy5"
   },
   "outputs": [],
   "source": [
    "scrubs2age = {\n",
    "     'Bob Kelso': 70,\n",
    "     'Carla Espinosa-Turk': 36,\n",
    "     'Christopher Turk': 31,\n",
    "     'Elliot Reid': 29,\n",
    "     'J.D.': 31,\n",
    "     'Janitor': 40,\n",
    "     'Perry Cox': 45\n",
    "  }\n",
    "\n",
    "scrubs2cred = {\n",
    "     'Bob Kelso': 'M.D.',\n",
    "     'Carla Espinosa-Turk': 'RN',\n",
    "     'Christopher Turk': 'M.D.',\n",
    "     'Elliot Reid': 'M.D.',\n",
    "     'J.D.': 'M.D.',\n",
    "     'Perry Cox': 'M.D.'\n",
    "  }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "XHHCJsVXmAy5"
   },
   "outputs": [],
   "source": [
    "# Your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aYXVFYyPmAy5"
   },
   "source": [
    "### Exercise 5\n",
    "\n",
    "A string is stored in the `coe_books` variable. \n",
    "\n",
    "* How many different books of Jonathan Coe are listed?\n",
    "* Check if the string starts with 'the accidental woman'\n",
    "* Print this text titlecased (every word starts with a capital letter, ignore the prepositions for now). And uppercased. And capitalized.\n",
    "* Convert the string to a list of words. Two of the book titles contain digits: 58 and 11. Find the index in the list and replace the string object for the number as integer. Print the list. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "S5b1g7tjmAy5",
    "tags": []
   },
   "outputs": [],
   "source": [
    "coe_books = \"\"\"the accidental woman\n",
    "a touch of love\n",
    "the dwarves of death\n",
    "what a carve up! or the winshaw legacy viking\n",
    "the house of sleep\n",
    "the rotters' club\n",
    "the closed circle\n",
    "the rain before it falls\n",
    "the terrible privacy of maxwell sim\n",
    "expo 58\n",
    "number 11\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "dey43Pi0mAy6"
   },
   "outputs": [],
   "source": [
    "# Your code here\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Z0ZI_N9ckwTM",
    "tags": []
   },
   "source": [
    "\n",
    "\n",
    "## Graded Assignment Week 2\n",
    "\n",
    "### To be announced on Canvas"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "name": "2_InputOutput.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}