Copy_of_tflite_c03_exercise_convert_model_to_tflite.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "Copy of tflite_c03_exercise_convert_model_to_tflite.ipynb",
      "provenance": [],
      "private_outputs": true,
      "collapsed_sections": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/jonbaer/googlecolab/blob/master/Copy_of_tflite_c03_exercise_convert_model_to_tflite.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Za8-Nr5k11fh"
      },
      "source": [
        "##### Copyright 2018 The TensorFlow Authors."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "colab_type": "code",
        "id": "Eq10uEbw0E4l",
        "colab": {}
      },
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "KlUrRaN4w3ct"
      },
      "source": [
        "# Train Your Own Model and Convert It to TFLite"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "H3UojxdNw8J1"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/examples/blob/master/courses/udacity_intro_to_tensorflow_lite/tflite_c03_exercise_convert_model_to_tflite.ipynb\">\n",
        "    <img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />\n",
        "    Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/tensorflow/examples/blob/master/courses/udacity_intro_to_tensorflow_lite/tflite_c03_exercise_convert_model_to_tflite.ipynb\">\n",
        "    <img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />\n",
        "    View source on GitHub</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "pXX-pi1r6NfG"
      },
      "source": [
        "\n",
        "\n",
        "This notebook uses the [Fashion MNIST](https://github.com/zalandoresearch/fashion-mnist) dataset which contains 70,000 grayscale images in 10 categories. The images show individual articles of clothing at low resolution (28 by 28 pixels), as seen here:\n",
        "\n",
        "<table>\n",
        "  <tr><td>\n",
        "    <img src=\"https://tensorflow.org/images/fashion-mnist-sprite.png\"\n",
        "         alt=\"Fashion MNIST sprite\"  width=\"600\">\n",
        "  </td></tr>\n",
        "  <tr><td align=\"center\">\n",
        "    <b>Figure 1.</b> <a href=\"https://github.com/zalandoresearch/fashion-mnist\">Fashion-MNIST samples</a> (by Zalando, MIT License).<br/>&nbsp;\n",
        "  </td></tr>\n",
        "</table>\n",
        "\n",
        "Fashion MNIST is intended as a drop-in replacement for the classic [MNIST](http://yann.lecun.com/exdb/mnist/) dataset—often used as the \"Hello, World\" of machine learning programs for computer vision. The MNIST dataset contains images of handwritten digits (0, 1, 2, etc.) in a format identical to that of the articles of clothing we'll use here.\n",
        "\n",
        "This uses Fashion MNIST for variety, and because it's a slightly more challenging problem than regular MNIST. Both datasets are relatively small and are used to verify that an algorithm works as expected. They're good starting points to test and debug code.\n",
        "\n",
        "We will use 60,000 images to train the network and 10,000 images to evaluate how accurately the network learned to classify images. You can access the Fashion MNIST directly from TensorFlow. Import and load the Fashion MNIST data directly from TensorFlow:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "rjOAfhgd__Sp"
      },
      "source": [
        "# Setup"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "Bp_nvHnh_tDo",
        "colab": {}
      },
      "source": [
        "try:\n",
        "  %tensorflow_version 2.x\n",
        "except:\n",
        "  pass"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "pfyZKowNAQ4j",
        "colab": {}
      },
      "source": [
        "from __future__ import absolute_import, division, print_function, unicode_literals\n",
        "\n",
        "# TensorFlow and tf.keras\n",
        "import tensorflow as tf\n",
        "from tensorflow import keras\n",
        "\n",
        "import tensorflow_datasets as tfds\n",
        "tfds.disable_progress_bar()\n",
        "\n",
        "# Helper libraries\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "import pathlib\n",
        "\n",
        "print(tf.__version__)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "tadPBTEiAprt"
      },
      "source": [
        "# Download Fashion MNIST Dataset\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "Ds9gfZKzAnkX",
        "colab": {}
      },
      "source": [
        "splits = tfds.Split.ALL.subsplit(weighted=(80, 10, 10))\n",
        "\n",
        "splits, info = tfds.load('fashion_mnist', with_info=True, as_supervised=True, split=splits)\n",
        "\n",
        "(train_examples, validation_examples, test_examples) = splits\n",
        "\n",
        "num_examples = info.splits['train'].num_examples\n",
        "num_classes = info.features['label'].num_classes"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "-eAv71FRm4JE",
        "colab": {}
      },
      "source": [
        "class_names = ['T-shirt_top', 'Trouser', 'Pullover', 'Dress', 'Coat',\n",
        "               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "hXe6jNokqX3_",
        "colab": {}
      },
      "source": [
        "with open('labels.txt', 'w') as f:\n",
        "  f.write('\\n'.join(class_names))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "q0RxpwTmQN-y",
        "colab": {}
      },
      "source": [
        "IMG_SIZE = 28"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "ZAkuq0V0Aw2X"
      },
      "source": [
        "# Preprocessing data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "_5SIivkunKCC"
      },
      "source": [
        "## Preprocess"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "nQMIkJf9AvJ4",
        "colab": {}
      },
      "source": [
        "# Write a function to normalize and resize the images\n",
        "\n",
        "def format_example(image, label):\n",
        "  # Cast image to float32\n",
        "  image = # YOUR CODE HERE\n",
        "  # Resize the image if necessary\n",
        "  image = # YOUR CODE HERE\n",
        "  # Normalize the image in the range [0, 1]\n",
        "  image = # YOUR CODE HERE\n",
        "  return image, label"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "oEQP743aMv4C",
        "colab": {}
      },
      "source": [
        "# Set the batch size to 32\n",
        "\n",
        "BATCH_SIZE = 32"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "JM4HfIJtnNEk"
      },
      "source": [
        "## Create a Dataset from images and labels"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "zOL4gSUARFjM",
        "colab": {}
      },
      "source": [
        "# Prepare the examples by preprocessing the them and then batching them (and optionally prefetching them)\n",
        "\n",
        "# If you wish you can shuffle train set here\n",
        "train_batches = # YOUR CODE HERE\n",
        "\n",
        "validation_batches = # YOUR CODE HERE\n",
        "test_batches = # YOUR CODE HERE"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "M-topQaOm_LM"
      },
      "source": [
        "# Building the model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "4gsYqdIlEFVg",
        "colab": {}
      },
      "source": [
        "\"\"\"\n",
        "Model: \"sequential\"\n",
        "_________________________________________________________________\n",
        "Layer (type)                 Output Shape              Param #   \n",
        "=================================================================\n",
        "conv2d (Conv2D)              (None, 26, 26, 16)        160       \n",
        "_________________________________________________________________\n",
        "max_pooling2d (MaxPooling2D) (None, 13, 13, 16)        0         \n",
        "_________________________________________________________________\n",
        "conv2d_1 (Conv2D)            (None, 11, 11, 32)        4640      \n",
        "_________________________________________________________________\n",
        "flatten (Flatten)            (None, 3872)              0         \n",
        "_________________________________________________________________\n",
        "dense (Dense)                (None, 64)                247872    \n",
        "_________________________________________________________________\n",
        "dense_1 (Dense)              (None, 10)                650       \n",
        "=================================================================\n",
        "Total params: 253,322\n",
        "Trainable params: 253,322\n",
        "Non-trainable params: 0\n",
        "\"\"\""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "kDqcwksFB1bh",
        "colab": {}
      },
      "source": [
        "# Build the model shown in the previous cell\n",
        "\n",
        "\n",
        "model = tf.keras.Sequential([\n",
        "  # Set the input shape to (28, 28, 1), kernel size=3, filters=16 and use ReLU activation,  \n",
        "  tf.keras.layers.Conv2D(# YOUR CODE HERE),    \n",
        "  tf.keras.layers.MaxPooling2D(),\n",
        "  # Set the number of filters to 32, kernel size to 3 and use ReLU activation \n",
        "  tf.keras.layers.Conv2D(# YOUR CODE HERE),\n",
        "  # Flatten the output layer to 1 dimension\n",
        "  tf.keras.layers.Flatten(),\n",
        "  # Add a fully connected layer with 64 hidden units and ReLU activation\n",
        "  tf.keras.layers.Dense(# YOUR CODE HERE),\n",
        "  # Attach a final softmax classification head\n",
        "  tf.keras.layers.Dense(# YOUR CODE HERE)])\n",
        "\n",
        "# Set the loss and accuracy metrics\n",
        "model.compile(\n",
        "    optimizer='adam', \n",
        "    loss=# YOUR CODE HERE, \n",
        "    metrics=# YOUR CODE HERE)\n",
        "      "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "zEMOz-LDnxgD"
      },
      "source": [
        "## Train"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "JGlNoRtzCP4_",
        "colab": {}
      },
      "source": [
        "model.fit(train_batches, \n",
        "          epochs=10,\n",
        "          validation_data=validation_batches)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "TZT9-7w9n4YO"
      },
      "source": [
        "# Exporting to TFLite"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "9dq78KBkCV2_",
        "colab": {}
      },
      "source": [
        "export_dir = 'saved_model/1'\n",
        "\n",
        "# Use the tf.saved_model API to export the SavedModel\n",
        "\n",
        "# Your Code Here"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "colab_type": "code",
        "id": "EDGiYrBdE6fl",
        "colab": {}
      },
      "source": [
        "#@title Select mode of optimization\n",
        "mode = \"Speed\" #@param [\"Default\", \"Storage\", \"Speed\"]\n",
        "\n",
        "if mode == 'Storage':\n",
        "  optimization = tf.lite.Optimize.OPTIMIZE_FOR_SIZE\n",
        "elif mode == 'Speed':\n",
        "  optimization = tf.lite.Optimize.OPTIMIZE_FOR_LATENCY\n",
        "else:\n",
        "  optimization = tf.lite.Optimize.DEFAULT"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "SLskPWHsG4Nj",
        "colab": {}
      },
      "source": [
        "optimization"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "RbcS9C00CzGe",
        "colab": {}
      },
      "source": [
        "# Use the TFLiteConverter SavedModel API to initialize the converter\n",
        "converter = # YOUR CODE HERE\n",
        "\n",
        "# Set the optimzations\n",
        "converter.optimizations = # YOUR CODE HERE\n",
        "\n",
        "# Invoke the converter to finally generate the TFLite model\n",
        "tflite_model = # YOUR CODE HERE"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "q5PWCDsTC3El",
        "colab": {}
      },
      "source": [
        "tflite_model_file = 'model.tflite'\n",
        "\n",
        "with open(tflite_model_file, \"wb\") as f:\n",
        "  f.write(tflite_model)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "SR6wFcQ1Fglm"
      },
      "source": [
        "# Test if your model is working"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "O3IFOcUEIzQx",
        "colab": {}
      },
      "source": [
        "# Load TFLite model and allocate tensors.\n",
        "interpreter = tf.lite.Interpreter(model_content=tflite_model)\n",
        "interpreter.allocate_tensors()\n",
        "\n",
        "input_index = interpreter.get_input_details()[0][\"index\"]\n",
        "output_index = interpreter.get_output_details()[0][\"index\"]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "rKcToCBEC-Bu",
        "colab": {}
      },
      "source": [
        "# Gather results for the randomly sampled test images\n",
        "predictions = []\n",
        "test_labels = []\n",
        "test_images = []\n",
        "\n",
        "for img, label in test_batches.take(50):\n",
        "  interpreter.set_tensor(input_index, img)\n",
        "  interpreter.invoke()\n",
        "  predictions.append(interpreter.get_tensor(output_index))\n",
        "  test_labels.append(label[0])\n",
        "  test_images.append(np.array(img))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "colab_type": "code",
        "id": "kSjTmi05Tyod",
        "colab": {}
      },
      "source": [
        "#@title Utility functions for plotting\n",
        "# Utilities for plotting\n",
        "\n",
        "def plot_image(i, predictions_array, true_label, img):\n",
        "  predictions_array, true_label, img = predictions_array[i], true_label[i], img[i]\n",
        "  plt.grid(False)\n",
        "  plt.xticks([])\n",
        "  plt.yticks([])\n",
        "  \n",
        "  img = np.squeeze(img)\n",
        "\n",
        "  plt.imshow(img, cmap=plt.cm.binary)\n",
        "\n",
        "  predicted_label = np.argmax(predictions_array)\n",
        "  if predicted_label == true_label.numpy():\n",
        "    color = 'green'\n",
        "  else:\n",
        "    color = 'red'\n",
        "    \n",
        "  plt.xlabel(\"{} {:2.0f}% ({})\".format(class_names[predicted_label],\n",
        "                                100*np.max(predictions_array),\n",
        "                                class_names[true_label]),\n",
        "                                color=color)\n",
        "\n",
        "def plot_value_array(i, predictions_array, true_label):\n",
        "  predictions_array, true_label = predictions_array[i], true_label[i]\n",
        "  plt.grid(False)\n",
        "  plt.xticks(list(range(10)), class_names, rotation='vertical')\n",
        "  plt.yticks([])\n",
        "  thisplot = plt.bar(range(10), predictions_array[0], color=\"#777777\")\n",
        "  plt.ylim([0, 1])\n",
        "  predicted_label = np.argmax(predictions_array[0])\n",
        "\n",
        "  thisplot[predicted_label].set_color('red')\n",
        "  thisplot[true_label].set_color('green')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "colab_type": "code",
        "id": "ZZwg0wFaVXhZ",
        "colab": {}
      },
      "source": [
        "#@title Visualize the outputs { run: \"auto\" }\n",
        "index = 49 #@param {type:\"slider\", min:1, max:50, step:1}\n",
        "plt.figure(figsize=(6,3))\n",
        "plt.subplot(1,2,1)\n",
        "plot_image(index, predictions, test_labels, test_images)\n",
        "plt.show()\n",
        "plot_value_array(index, predictions, test_labels)\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "076bo3FMpRDb"
      },
      "source": [
        "# Download TFLite model and assets\n",
        "\n",
        "**NOTE: You might have to run to the cell below twice**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "XsPXqPlgZPjE",
        "colab": {}
      },
      "source": [
        "try:\n",
        "  from google.colab import files\n",
        "  files.download(tflite_model_file)\n",
        "  files.download('labels.txt')\n",
        "except:\n",
        "  pass"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "VyBVNwAzH3Oe"
      },
      "source": [
        "# Deploying TFLite model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "pdfa5L6wH87u"
      },
      "source": [
        "Now once you've the trained TFLite model downloaded, you can ahead and deploy this on an Android/iOS application by placing the model assets in the appropriate location."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "iLY6X8P90L0P"
      },
      "source": [
        "# Prepare the test images for download (Optional)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "G3bjzLj10OJv",
        "colab": {}
      },
      "source": [
        "!mkdir -p test_images"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "pVrBZv1-0Py-",
        "colab": {}
      },
      "source": [
        "from PIL import Image\n",
        "\n",
        "for index, (image, label) in enumerate(test_batches.take(50)):\n",
        "  image = tf.cast(image * 255.0, tf.uint8)\n",
        "  image = tf.squeeze(image).numpy()\n",
        "  pil_image = Image.fromarray(image)\n",
        "  pil_image.save('test_images/{}_{}.jpg'.format(class_names[label[0]].lower(), index))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "nX0N0M8u0R2s",
        "colab": {}
      },
      "source": [
        "!ls test_images"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "LvLht1QM0W8k",
        "colab": {}
      },
      "source": [
        "!zip -qq fmnist_test_images.zip -r test_images/"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "FdOq-4sT0X95",
        "colab": {}
      },
      "source": [
        "try:\n",
        "  files.download('fmnist_test_images.zip')\n",
        "except:\n",
        "  pass"
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}