TensorFlow: Advanced Techniques/C3_W1_Lab_1_transfer_learning_cats_dogs.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "C3_W1_Lab_1_transfer_learning_cats_dogs.ipynb",
      "private_outputs": true,
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.4"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fYJqjq66JVQQ"
      },
      "source": [
        "# Basic transfer learning with cats and dogs data\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0oWuHhhcJVQQ"
      },
      "source": [
        "### Import tensorflow"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ioLbtB3uGKPX"
      },
      "source": [
        "try:\n",
        "  # %tensorflow_version only exists in Colab.\n",
        "  %tensorflow_version 2.x\n",
        "except Exception:\n",
        "  pass"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gjfMJAHPJVQR"
      },
      "source": [
        "### Import modules and download the cats and dogs dataset."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "y23ucAFLoHop"
      },
      "source": [
        "import urllib.request\n",
        "import os\n",
        "import zipfile\n",
        "import random\n",
        "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
        "from tensorflow.keras import layers\n",
        "from tensorflow.keras import Model\n",
        "from tensorflow.keras.applications.inception_v3 import InceptionV3\n",
        "from tensorflow.keras.optimizers import RMSprop\n",
        "from shutil import copyfile\n",
        "\n",
        "\n",
        "data_url = \"https://download.microsoft.com/download/3/E/1/3E1C3F21-ECDB-4869-8368-6DEBA77B919F/kagglecatsanddogs_3367a.zip\"\n",
        "data_file_name = \"catsdogs.zip\"\n",
        "download_dir = '/tmp/'\n",
        "urllib.request.urlretrieve(data_url, data_file_name)\n",
        "zip_ref = zipfile.ZipFile(data_file_name, 'r')\n",
        "zip_ref.extractall(download_dir)\n",
        "zip_ref.close()\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JNVXCUNUJVQR"
      },
      "source": [
        "Check that the dataset has the expected number of examples."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AwMoZHxWOynx"
      },
      "source": [
        "print(\"Number of cat images:\",len(os.listdir('/tmp/PetImages/Cat/')))\n",
        "print(\"Number of dog images:\", len(os.listdir('/tmp/PetImages/Dog/')))\n",
        "\n",
        "# Expected Output:\n",
        "# Number of cat images: 12501\n",
        "# Number of dog images: 12501"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_0riaptkJVQR"
      },
      "source": [
        "Create some folders that will store the training and test data.\n",
        "- There will be a training folder and a testing folder.\n",
        "- Each of these will have a subfolder for cats and another subfolder for dogs."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qygIo4W5O1hQ"
      },
      "source": [
        "try:\n",
        "    os.mkdir('/tmp/cats-v-dogs')\n",
        "    os.mkdir('/tmp/cats-v-dogs/training')\n",
        "    os.mkdir('/tmp/cats-v-dogs/testing')\n",
        "    os.mkdir('/tmp/cats-v-dogs/training/cats')\n",
        "    os.mkdir('/tmp/cats-v-dogs/training/dogs')\n",
        "    os.mkdir('/tmp/cats-v-dogs/testing/cats')\n",
        "    os.mkdir('/tmp/cats-v-dogs/testing/dogs')\n",
        "except OSError:\n",
        "    pass"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1ZHD_c-sJVQR"
      },
      "source": [
        "### Split data into training and test sets\n",
        "\n",
        "- The following code put first checks if an image file is empty (zero length)\n",
        "- Of the files that are not empty, it puts 90% of the data into the training set, and 10% into the test set."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "M90EiIu0O314"
      },
      "source": [
        "import random\n",
        "from shutil import copyfile\n",
        "def split_data(SOURCE, TRAINING, TESTING, SPLIT_SIZE):\n",
        "    files = []\n",
        "    for filename in os.listdir(SOURCE):\n",
        "        file = SOURCE + filename\n",
        "        if os.path.getsize(file) > 0:\n",
        "            files.append(filename)\n",
        "        else:\n",
        "            print(filename + \" is zero length, so ignoring.\")\n",
        "\n",
        "    training_length = int(len(files) * SPLIT_SIZE)\n",
        "    testing_length = int(len(files) - training_length)\n",
        "    shuffled_set = random.sample(files, len(files))\n",
        "    training_set = shuffled_set[0:training_length]\n",
        "    testing_set = shuffled_set[training_length:]\n",
        "\n",
        "    for filename in training_set:\n",
        "        this_file = SOURCE + filename\n",
        "        destination = TRAINING + filename\n",
        "        copyfile(this_file, destination)\n",
        "\n",
        "    for filename in testing_set:\n",
        "        this_file = SOURCE + filename\n",
        "        destination = TESTING + filename\n",
        "        copyfile(this_file, destination)\n",
        "\n",
        "\n",
        "CAT_SOURCE_DIR = \"/tmp/PetImages/Cat/\"\n",
        "TRAINING_CATS_DIR = \"/tmp/cats-v-dogs/training/cats/\"\n",
        "TESTING_CATS_DIR = \"/tmp/cats-v-dogs/testing/cats/\"\n",
        "DOG_SOURCE_DIR = \"/tmp/PetImages/Dog/\"\n",
        "TRAINING_DOGS_DIR = \"/tmp/cats-v-dogs/training/dogs/\"\n",
        "TESTING_DOGS_DIR = \"/tmp/cats-v-dogs/testing/dogs/\"\n",
        "\n",
        "split_size = .9\n",
        "split_data(CAT_SOURCE_DIR, TRAINING_CATS_DIR, TESTING_CATS_DIR, split_size)\n",
        "split_data(DOG_SOURCE_DIR, TRAINING_DOGS_DIR, TESTING_DOGS_DIR, split_size)\n",
        "\n",
        "# Expected output\n",
        "# 666.jpg is zero length, so ignoring\n",
        "# 11702.jpg is zero length, so ignoring"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KMx_pePuJVQR"
      },
      "source": [
        "Check that the training and test sets are the expected lengths."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cl8sQpM1O9xK"
      },
      "source": [
        "\n",
        "print(\"Number of training cat images\", len(os.listdir('/tmp/cats-v-dogs/training/cats/')))\n",
        "print(\"Number of training dog images\", len(os.listdir('/tmp/cats-v-dogs/training/dogs/')))\n",
        "print(\"Number of testing cat images\", len(os.listdir('/tmp/cats-v-dogs/testing/cats/')))\n",
        "print(\"Number of testing dog images\", len(os.listdir('/tmp/cats-v-dogs/testing/dogs/')))\n",
        "\n",
        "# expected output\n",
        "# Number of training cat images 11250\n",
        "# Number of training dog images 11250\n",
        "# Number of testing cat images 1250\n",
        "# Number of testing dog images 1250"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pNz89__rJVQR"
      },
      "source": [
        "### Data augmentation (try adjusting the parameters)!\n",
        "\n",
        "Here, you'll use the `ImageDataGenerator` to perform data augmentation.  \n",
        "- Things like rotating and flipping the existing images allows you to generate training data that is more varied, and can help the model generalize better during training.  \n",
        "- You can also use the data generator to apply data augmentation to the validation set.\n",
        "\n",
        "You can use the default parameter values for a first pass through this lab.\n",
        "- Later, try to experiment with the parameters of `ImageDataGenerator` to improve the model's performance.\n",
        "- Try to drive reach 99.9% validation accuracy or better."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TVO1l8vAPE14"
      },
      "source": [
        "\n",
        "TRAINING_DIR = \"/tmp/cats-v-dogs/training/\"\n",
        "# Experiment with your own parameters to reach 99.9% validation accuracy or better\n",
        "train_datagen = ImageDataGenerator(rescale=1./255,\n",
        "      rotation_range=40,\n",
        "      width_shift_range=0.2,\n",
        "      height_shift_range=0.2,\n",
        "      shear_range=0.2,\n",
        "      zoom_range=0.2,\n",
        "      horizontal_flip=True,\n",
        "      fill_mode='nearest')\n",
        "train_generator = train_datagen.flow_from_directory(TRAINING_DIR,\n",
        "                                                    batch_size=100,\n",
        "                                                    class_mode='binary',\n",
        "                                                    target_size=(150, 150))\n",
        "\n",
        "VALIDATION_DIR = \"/tmp/cats-v-dogs/testing/\"\n",
        "\n",
        "validation_datagen = ImageDataGenerator(rescale=1./255)\n",
        "validation_generator = validation_datagen.flow_from_directory(VALIDATION_DIR,\n",
        "                                                              batch_size=100,\n",
        "                                                              class_mode='binary',\n",
        "                                                              target_size=(150, 150))\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dry-R5OT6N11"
      },
      "source": [
        "train_generator"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WchwDzWNJVQR"
      },
      "source": [
        "### Get and prepare the model\n",
        "\n",
        "You'll be using the `InceptionV3` model.  \n",
        "- Since you're making use of transfer learning, you'll load the pre-trained weights of the model.\n",
        "- You'll also freeze the existing layers so that they aren't trained on your downstream task with the cats and dogs data.\n",
        "- You'll also get a reference to the last layer, 'mixed7' because you'll add some layers after this last layer."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tiPK1LlMOvm7"
      },
      "source": [
        "weights_url = \"https://storage.googleapis.com/mledu-datasets/inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5\"\n",
        "weights_file = \"inception_v3.h5\"\n",
        "urllib.request.urlretrieve(weights_url, weights_file)\n",
        "\n",
        "# Instantiate the model\n",
        "pre_trained_model = InceptionV3(input_shape=(150, 150, 3),\n",
        "                                include_top=False,\n",
        "                                weights=None)\n",
        "\n",
        "# load pre-trained weights\n",
        "pre_trained_model.load_weights(weights_file)\n",
        "\n",
        "# freeze the layers\n",
        "for layer in pre_trained_model.layers:\n",
        "    layer.trainable = False\n",
        "\n",
        "# pre_trained_model.summary()\n",
        "\n",
        "last_layer = pre_trained_model.get_layer('mixed7')\n",
        "print('last layer output shape: ', last_layer.output_shape)\n",
        "last_output = last_layer.output\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E-R4v2zx65nL"
      },
      "source": [
        "from keras.utils import plot_model\n",
        "plot_model(pre_trained_model)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3edBz_IxJVQR"
      },
      "source": [
        "### Add layers\n",
        "Add some layers that you will train on the cats and dogs data.\n",
        "- `Flatten`: This will take the output of the `last_layer` and flatten it to a vector.\n",
        "- `Dense`: You'll add a dense layer with a relu activation.\n",
        "- `Dense`: After that, add a dense layer with a sigmoid activation.  The sigmoid will scale the output to range from 0 to 1, and allow you to interpret the output as a prediction between two categories (cats or dogs).\n",
        "\n",
        "Then create the model object."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oDidHXO1JVQR"
      },
      "source": [
        "# Flatten the output layer to 1 dimension\n",
        "x = layers.Flatten()(last_output)\n",
        "# Add a fully connected layer with 1,024 hidden units and ReLU activation\n",
        "x = layers.Dense(1024, activation='relu')(x)\n",
        "# Add a final sigmoid layer for classification\n",
        "x = layers.Dense(1, activation='sigmoid')(x)\n",
        "\n",
        "model = Model(pre_trained_model.input, x)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5tijUhxqJtcj"
      },
      "source": [
        "model.summary()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "asCm8okXJVQR"
      },
      "source": [
        "### Train the model\n",
        "Compile the model, and then train it on the test data using `model.fit`\n",
        "- Feel free to adjust the number of epochs.  This project was originally designed with 20 epochs.\n",
        "- For the sake of time, you can use fewer epochs (2) to see how the code runs.\n",
        "- You can ignore the warnings about some of the images having corrupt EXIF data. Those will be skipped."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3nxUncKWPRhR"
      },
      "source": [
        "\n",
        "# compile the model\n",
        "model.compile(optimizer=RMSprop(lr=0.0001),\n",
        "              loss='binary_crossentropy',\n",
        "              metrics=['acc'])\n",
        "\n",
        "# train the model (adjust the number of epochs from 1 to improve performance)\n",
        "history = model.fit(\n",
        "            train_generator,\n",
        "            validation_data=validation_generator,\n",
        "            epochs=2,\n",
        "            verbose=1)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H6Oo6kM-JVQR"
      },
      "source": [
        "### Visualize the training and validation accuracy\n",
        "\n",
        "You can see how the training and validation accuracy change with each epoch on an x-y plot."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "erDopoQ5eNL7"
      },
      "source": [
        "%matplotlib inline\n",
        "\n",
        "import matplotlib.image  as mpimg\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "#-----------------------------------------------------------\n",
        "# Retrieve a list of list results on training and test data\n",
        "# sets for each training epoch\n",
        "#-----------------------------------------------------------\n",
        "acc=history.history['acc']\n",
        "val_acc=history.history['val_acc']\n",
        "loss=history.history['loss']\n",
        "val_loss=history.history['val_loss']\n",
        "\n",
        "epochs=range(len(acc)) # Get number of epochs\n",
        "\n",
        "#------------------------------------------------\n",
        "# Plot training and validation accuracy per epoch\n",
        "#------------------------------------------------\n",
        "plt.plot(epochs, acc, 'r', \"Training Accuracy\")\n",
        "plt.plot(epochs, val_acc, 'b', \"Validation Accuracy\")\n",
        "plt.title('Training and validation accuracy')\n",
        "plt.figure()\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xKc_1Qm8JVQR"
      },
      "source": [
        "### Predict on a test image\n",
        "\n",
        "You can upload any image and have the model predict whether it's a dog or a cat.\n",
        "- Find an image of a dog or cat\n",
        "- Run the following code cell.  It will ask you to upload an image.\n",
        "- The model will print \"is a dog\" or \"is a cat\" depending on the model's prediction."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_0R9fsf4w29e"
      },
      "source": [
        "import numpy as np\n",
        "from google.colab import files\n",
        "from keras.preprocessing import image\n",
        "\n",
        "uploaded = files.upload()\n",
        "\n",
        "for fn in uploaded.keys():\n",
        " \n",
        "  # predicting images\n",
        "  path = '/content/' + fn\n",
        "  img = image.load_img(path, target_size=(150, 150))\n",
        "  x = image.img_to_array(img)\n",
        "  x = np.expand_dims(x, axis=0)\n",
        "\n",
        "  image_tensor = np.vstack([x])\n",
        "  classes = model.predict(image_tensor)\n",
        "  print(classes)\n",
        "  print(classes[0])\n",
        "  if classes[0]>0.5:\n",
        "    print(fn + \" is a dog\")\n",
        "  else:\n",
        "    print(fn + \" is a cat\")"
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}