Ex2.Classifiying-MNist.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<image src=\"https://www.th-koeln.de/img/logo.svg\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://www.th-koeln.de/img/logo.svg\" style=\"float: right;\" width=\"200\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2nd exercise: <font color=\"#C70039\">Classifying the MNIST data set</font>\n",
    "* Course: DIS21a.1\n",
    "* Lecturer: <a href=\"https://www.gernotheisenberg.de/\">Gernot Heisenberg</a>\n",
    "* Author of notebook modifications and adaptations: <a href=\"https://www.gernotheisenberg.de/\">Gernot Heisenberg</a>\n",
    "* Date:   01.04.2020\n",
    "\n",
    "<img src=\"https://upload.wikimedia.org/wikipedia/commons/2/27/MnistExamples.png\" style=\"float: center;\" width=\"450\">\n",
    "\n",
    "---------------------------------\n",
    "**GENERAL NOTE 1**: \n",
    "Please make sure you are reading the entire notebook, since it contains a lot of information on your tasks (e.g. regarding the set of certain paramaters or a specific computational trick), and the written mark downs as well as comments contain a lot of information on how things work together as a whole. \n",
    "\n",
    "**GENERAL NOTE 2**: \n",
    "* Please, when commenting source code, just use English language only. \n",
    "* When describing an observation (for instance, after you have run through your test plan) you may use German language.\n",
    "This applies to all exercises in DIS 21a.1.  \n",
    "\n",
    "---------------------\n",
    "\n",
    "### <font color=\"ce33ff\">DESCRIPTION</font>:\n",
    "This notebook allows you for classifying the MNIST data set. That is an inbuild data set coming along with keras. Keras is using tensorflow in the backend.\n",
    "\n",
    "-------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "### <font color=\"FFC300\">TASKS</font>:\n",
    "The tasks that you need to work on in this notebook are always indicated below as bullet points. \n",
    "If a task is more challenging and consists of several steps, this is indicated as well. \n",
    "Make sure you have worked down the task list and commented your doings. \n",
    "This should be done by using markdown.<br> \n",
    "<font color=red>Make sure you don't forget to specify your name and your matriculation number in the notebook before submitting it.</font>\n",
    "\n",
    "**YOUR TASKS in this exercise are as follows**:\n",
    "1. import the notebook to Google Colab.\n",
    "2. make sure you specified you name and your matriculation number in the header below my name and date. \n",
    "    * set the date too and remove mine.\n",
    "3. read the *entire* notebook carefully \n",
    "    * add comments whereever you feel it necessary for better understanding\n",
    "    * run the notebook for the first time and note the result in a markdown table. \n",
    "        * I have provided you with an example of a table in markdown (see below). Make sure you adapt your table accordingly. \n",
    "        * Put the table at the end of the notebook. \n",
    "        * This type of table will be needed in the other exercises as well. Always put it at the end.\n",
    "    \n",
    "| type of method | loss function | optimizer | accuracy |\n",
    "| :-: | :-: | :-: | :-: |\n",
    "| classification | categorical_crossentropy | bamm !|.666\n",
    "\n",
    "4. do some tensor slicing on the training images as well as the labels\n",
    "    * slice the first 100\n",
    "    * slice the last 13\n",
    "    * slice between 666 and 1000\n",
    "    * slice one image at index '666' and slice the center part of that image \n",
    "        * visualize that image (original and sliced)\n",
    "5. take less training data and rerun the network\n",
    "    * add the size of the training data as a column in the table and note the accuracy you achieve\n",
    "6. take less testing data and rerun the network\n",
    "    * add the size of the testing data as a column in the table and note the accuracy you achieve\n",
    "7. increase/decrease the number of epochs and the batch size \n",
    "    * add those hyperparameters as columns in the table and note the accuracy you achieve\n",
    "8. make combinations of this (NOTE: what you are doing here is writing your own test plan, a big future task of yours once you became a data scientist. Make sure you combine with sense and intelligence and not just chaotically.)\n",
    "9. comment your observations\n",
    "    * when is the accuracy increasing / decreasing\n",
    "\n",
    "-----------------------------------------------------------------------------------\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## START OF THE NOTEBOOK CODE\n",
    "----------------------------------------------------------------------------------------------------------------------\n",
    "### necessary imports\n",
    "others are going to be included as soon as they are needed\n",
    "\n",
    "**GENERAL CODE REMARK**: the common way to import is: \"from keras.datasets import mnist\". (see below) \n",
    "However, this throws several warnings depending on the version of tensorflow you are trying to use. Sometimes this is not compatible.<br>\n",
    "However, by adding \"tensorflow.\" before \"keras.\" that warning is resolved.  \n",
    "This hack was found in a thread on stackoverflow, that is a valuable ressource for you too."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras.datasets import mnist\n",
    "\n",
    "'''Original'''\n",
    "# from keras.datasets import mnist\n",
    "# if you are using Google Colab, this import line should work without causing any problems. \n",
    "# See general code remark above"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### loading the MNIST data set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### print some shape of the training images and labels until you feel comfortable with the data set structure"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_images.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_labels.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(test_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### more imports\n",
    "<font color=red>NOTE</font>: the original import was: \"from keras import models\" and \"layers\". However, this throws several warning. Adding \"tensorflow.\" before \"keras.\" solves this. // Found in a thread on stackoverflow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras import models\n",
    "from tensorflow.keras import layers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### building the ANN\n",
    "* set the activation function to ReLu\n",
    "* define the shape of the input images\n",
    "* define the size of the output layer\n",
    "* define the size of the hidden, fully connected layer\n",
    "* define softmax as the probability function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "network = models.Sequential()\n",
    "network.add(layers.Dense(512, activation='relu', input_shape=(28 * 28,)))\n",
    "network.add(layers.Dense(10, activation='softmax'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "network.compile(optimizer='rmsprop',loss='categorical_crossentropy',metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prepare the image data and normalize them (/255 since they were 8Bit-encoded)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_images = train_images.reshape((60000, 28 * 28))\n",
    "train_images = train_images.astype('float32') / 255\n",
    "\n",
    "test_images = test_images.reshape((10000, 28 * 28))\n",
    "test_images = test_images.astype('float32') / 255"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### categorical encoding of the labels is needed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras.utils import to_categorical\n",
    "\n",
    "train_labels = to_categorical(train_labels)\n",
    "test_labels = to_categorical(test_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "network.fit(train_images, train_labels, epochs=5, batch_size=128)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_loss, test_acc = network.evaluate(test_images, test_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('test_acc:', test_acc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's the final accuracy result for the classification of the test data set. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font color=\"#C70039\">Include your result table here and reflect a good test plan (see task list)</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}