notebooks/official/automl/automl_image_object_detection_export_edge.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "copyright"
      },
      "outputs": [],
      "source": [
        "# Copyright 2021 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "title"
      },
      "source": [
        "# AutoML training image object detection model for export to edge\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl_image_object_detection_export_edge.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"><br> Open in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fvertex-ai-samples%2Fmain%2Fnotebooks%2Fofficial%2Fautoml%2Fautoml_image_object_detection_export_edge.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://cloud.google.com/ml-engine/images/colab-enterprise-logo-32px.png\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
        "    </a>\n",
        "  </td>  \n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/automl_image_object_detection_export_edge.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\"><br>\n",
        "      View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/automl//automl_image_object_detection_export_edge.ipynb\">\n",
        "       <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br>\n",
        "      Open in Vertex AI Workbench\n",
        "    </a>\n",
        "  </td>\n",
        "</table>\n",
        "<br/><br/><br/>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "overview:automl,export_edge"
      },
      "source": [
        "## Overview\n",
        "\n",
        "\n",
        "This tutorial demonstrates how to use the Vertex AI SDK to create image object detection models to export as an Edge model using an AutoML model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "objective:automl,training,export_edge"
      },
      "source": [
        "### Objective\n",
        "\n",
        "In this tutorial, you create an AutoML image object detection model from a Python script using the Vertex SDK, and then export the model as an Edge model in TFLite format. You can alternatively create models with AutoML using the `gcloud` command-line tool or online using the Cloud Console.\n",
        "\n",
        "This tutorial uses the following Google Cloud Vertex AI services:\n",
        "\n",
        "- Vertex AI datasets\n",
        "- AutoML image\n",
        "\n",
        "The steps performed include:\n",
        "\n",
        "- Create a Vertex dataset resource.\n",
        "- Train the model.\n",
        "- Export the edge model from the model resource to Cloud Storage.\n",
        "- Download the model locally.\n",
        "- Make a local prediction."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dataset:salads,iod"
      },
      "source": [
        "### Dataset\n",
        "\n",
        "The dataset used for this tutorial is the Salads category of the [OpenImages dataset](https://www.tensorflow.org/datasets/catalog/open_images_v4) from [TensorFlow Datasets](https://www.tensorflow.org/datasets/catalog/overview). This dataset does not require any feature engineering. The version of the dataset you will use in this tutorial is stored in a public Cloud Storage bucket. The trained model predicts the bounding box locations and corresponding type of salad items in an image from a class of five items: salad, seafood, tomato, baked goods, or cheese."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "costs"
      },
      "source": [
        "### Costs\n",
        "\n",
        "This tutorial uses billable components of Google Cloud:\n",
        "\n",
        "* Vertex AI\n",
        "* Cloud Storage\n",
        "\n",
        "Learn about [Vertex AI\n",
        "pricing](https://cloud.google.com/vertex-ai/pricing) and [Cloud Storage\n",
        "pricing](https://cloud.google.com/storage/pricing), and use the [Pricing\n",
        "Calculator](https://cloud.google.com/products/calculator/)\n",
        "to generate a cost estimate based on your projected usage."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f0316df526f8"
      },
      "source": [
        "## Get started"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a2c2cb2109a0"
      },
      "source": [
        "### Install Vertex AI SDK for Python and other required packages\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6dca41de7a4d"
      },
      "outputs": [],
      "source": [
        "! pip3 install --upgrade --quiet google-cloud-aiplatform tensorflow gcsfs"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ff555b32bab8"
      },
      "source": [
        "### Restart runtime (Colab only)\n",
        "\n",
        "To use the newly installed packages, you must restart the runtime on Google Colab."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "f09b4dff629a"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "\n",
        "    import IPython\n",
        "\n",
        "    app = IPython.Application.instance()\n",
        "    app.kernel.do_shutdown(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ee775571c2b5"
      },
      "source": [
        "<div class=\"alert alert-block alert-warning\">\n",
        "<b>⚠️ The kernel is going to restart. Wait until it's finished before continuing to the next step. ⚠️</b>\n",
        "</div>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "92e68cfc3a90"
      },
      "source": [
        "### Authenticate your notebook environment (Colab only)\n",
        "\n",
        "Authenticate your environment on Google Colab.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "46604f70e831"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "\n",
        "    from google.colab import auth\n",
        "\n",
        "    auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4f872cd812d0"
      },
      "source": [
        "### Set Google Cloud project information and initialize Vertex AI SDK for Python\n",
        "\n",
        "To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com). Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "294fe4e5a671"
      },
      "outputs": [],
      "source": [
        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
        "LOCATION = \"us-central1\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bucket:mbsdk"
      },
      "source": [
        "### Create a Cloud Storage bucket\n",
        "\n",
        "Create a storage bucket to store intermediate artifacts such as datasets."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bucket"
      },
      "outputs": [],
      "source": [
        "BUCKET_URI = f\"gs://your-bucket-name-{PROJECT_ID}-unique\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_bucket"
      },
      "source": [
        "**If your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "create_bucket"
      },
      "outputs": [],
      "source": [
        "! gsutil mb -l $LOCATION $BUCKET_URI"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "init_aip:mbsdk"
      },
      "source": [
        "### Initialize Vertex AI SDK for Python\n",
        "\n",
        "Initialize the Vertex AI SDK for Python for your project and corresponding bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "init_aip:mbsdk"
      },
      "outputs": [],
      "source": [
        "from google.cloud import aiplatform\n",
        "\n",
        "aiplatform.init(project=PROJECT_ID, location=LOCATION, staging_bucket=BUCKET_URI)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tutorial_start:automl"
      },
      "source": [
        "# Tutorial\n",
        "\n",
        "Now you are ready to start creating your own AutoML image object detection model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "import_file:u_dataset,csv"
      },
      "source": [
        "#### Location of Cloud Storage training data.\n",
        "\n",
        "Now set the variable `IMPORT_FILE` to the location of the CSV index file in Cloud Storage."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "import_file:salads,csv,iod"
      },
      "outputs": [],
      "source": [
        "IMPORT_FILE = \"gs://cloud-samples-data/vision/salads.csv\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f41a55981d90"
      },
      "source": [
        "### Copying data between Google Cloud Storage Buckets \n",
        "\n",
        "In this step, you prevent access issues for the images in your original dataset. The code below extracts folder paths from image paths, constructs destination paths for Google Cloud Storage (GCS), copies images using gsutil commands, updates image paths in the DataFrame, and finally saves the modified DataFrame back to GCS as a CSV file."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "df98442ace03"
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "\n",
        "# Read the CSV file\n",
        "df = pd.read_csv(IMPORT_FILE, header=None)\n",
        "\n",
        "# Extract folder paths from image paths\n",
        "df[\"folder_path\"] = df.iloc[:, 0].apply(lambda x: \"/\".join(x.split(\"/\")[:-1]))\n",
        "\n",
        "# Construct destination paths in your bucket (adding a trailing slash for directories)\n",
        "df[\"destination_path\"] = (\n",
        "    BUCKET_URI\n",
        "    + \"/img/openimage/\"\n",
        "    + df[\"folder_path\"].apply(lambda x: x.split(\"/\")[-1])\n",
        "    + \"/\"\n",
        ")\n",
        "\n",
        "# Copy images using gsutil commands directly\n",
        "for src, dest in zip(df.iloc[:, 0], df[\"destination_path\"]):\n",
        "    ! gsutil -m cp {src} {dest}\n",
        "\n",
        "print(f\"Files copied to {BUCKET_URI}\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7ca1626de99f"
      },
      "outputs": [],
      "source": [
        "# Combine the destination folder paths with the original image filenames\n",
        "df[\"new_image_path\"] = df[\"destination_path\"] + df.iloc[:, 0].apply(\n",
        "    lambda x: x.split(\"/\")[-1]\n",
        ")\n",
        "\n",
        "# Replace the original image path column with the new full paths\n",
        "df.iloc[:, 0] = df[\"new_image_path\"]\n",
        "\n",
        "# Drop the temporary columns\n",
        "df = df.drop(columns=[\"new_image_path\", \"destination_path\", \"folder_path\"])\n",
        "\n",
        "# Specify the destination file path in your bucket for the updated CSV\n",
        "CSV_DESTINATION_PATH = f\"{BUCKET_URI}/vision/salads.csv\"\n",
        "\n",
        "# Save the updated DataFrame directly to GCS\n",
        "df.to_csv(CSV_DESTINATION_PATH, index=False, header=None)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ecc97105a2d7"
      },
      "source": [
        "#### Location of Cloud Storage training data.\n",
        "\n",
        "Redefining the variable `IMPORT_FILE` to the location of the CSV index file in Cloud Storage."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "028fb6ec54e0"
      },
      "outputs": [],
      "source": [
        "IMPORT_FILE = CSV_DESTINATION_PATH\n",
        "\n",
        "print(IMPORT_FILE)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f4fd562be838"
      },
      "source": [
        "#### Quick peek at your data\n",
        "\n",
        "This tutorial uses a version of salads dataset which is copied to the project's Cloud Storage Bucket.\n",
        "\n",
        "Start by doing a quick peek at the data. You count the number of examples by counting the number of rows in the CSV index file  (`wc -l`) and then peek at the first few rows."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1db4b3d511a6"
      },
      "outputs": [],
      "source": [
        "if \"IMPORT_FILES\" in globals():\n",
        "    FILE = IMPORT_FILES[0]\n",
        "else:\n",
        "    FILE = IMPORT_FILE\n",
        "\n",
        "count = ! gsutil cat $FILE | wc -l\n",
        "print(\"Number of Examples\", int(count[0]))\n",
        "\n",
        "print(\"First 10 rows\")\n",
        "! gsutil cat $FILE | head"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_dataset:image,iod"
      },
      "source": [
        "### Create the Dataset\n",
        "\n",
        "Next, create the dataset resource using the `create` method for the `ImageDataset` class, which takes the following parameters:\n",
        "\n",
        "- `display_name`: The human readable name for the dataset resource.\n",
        "- `gcs_source`: A list of one or more dataset index files to import the data items into the dataset resource.\n",
        "- `import_schema_uri`: The data labeling schema for the data items.\n",
        "\n",
        "This operation may take several minutes."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "create_dataset:image,iod"
      },
      "outputs": [],
      "source": [
        "dataset = aiplatform.ImageDataset.create(\n",
        "    display_name=\"Salads\",\n",
        "    gcs_source=[IMPORT_FILE],\n",
        "    import_schema_uri=aiplatform.schema.dataset.ioformat.image.bounding_box,\n",
        ")\n",
        "\n",
        "print(dataset.resource_name)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_automl_pipeline:image,edge,iod"
      },
      "source": [
        "### Create and run training pipeline\n",
        "\n",
        "To train an AutoML model, you perform two steps: 1) create a training pipeline, and 2) run the pipeline.\n",
        "\n",
        "#### Create training pipeline\n",
        "\n",
        "An AutoML training pipeline is created with the `AutoMLImageTrainingJob` class, with the following parameters:\n",
        "\n",
        "- `display_name`: The human readable name for the TrainingJob resource.\n",
        "- `prediction_type`: The type task to train the model for.\n",
        "  - `classification`: An image classification model.\n",
        "  - `object_detection`: An image object detection model.\n",
        "- `multi_label`: If a classification task, whether single (`False`) or multi-labeled (`True`).\n",
        "- `model_type`: The type of model for deployment.\n",
        "  - `CLOUD`: Deployment on Google Cloud\n",
        "  - `CLOUD_HIGH_ACCURACY_1`: Optimized for accuracy over latency for deployment on Google Cloud.\n",
        "  - `CLOUD_LOW_LATENCY_`: Optimized for latency over accuracy for deployment on Google Cloud.\n",
        "  - `MOBILE_TF_VERSATILE_1`: Deployment on an edge device.\n",
        "  - `MOBILE_TF_HIGH_ACCURACY_1`:Optimized for accuracy over latency for deployment on an edge device.\n",
        "  - `MOBILE_TF_LOW_LATENCY_1`: Optimized for latency over accuracy for deployment on an edge device.\n",
        "- `base_model`: (optional) Transfer learning from existing model resource -- supported for image classification only.\n",
        "\n",
        "The instantiated object is the DAG (directed acyclic graph) for the training job."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "create_automl_pipeline:image,edge,iod"
      },
      "outputs": [],
      "source": [
        "dag = aiplatform.AutoMLImageTrainingJob(\n",
        "    display_name=\"salads\",\n",
        "    prediction_type=\"object_detection\",\n",
        "    multi_label=False,\n",
        "    model_type=\"MOBILE_TF_LOW_LATENCY_1\",\n",
        "    base_model=None,\n",
        ")\n",
        "\n",
        "print(dag)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "run_automl_pipeline:image"
      },
      "source": [
        "#### Run the training pipeline\n",
        "\n",
        "Next, you run the DAG to start the training job by invoking the method `run`, with the following parameters:\n",
        "\n",
        "- `dataset`: The dataset resource to train the model.\n",
        "- `model_display_name`: The human readable name for the trained model.\n",
        "- `training_fraction_split`: The percentage of the dataset to use for training.\n",
        "- `test_fraction_split`: The percentage of the dataset to use for test (holdout data).\n",
        "- `validation_fraction_split`: The percentage of the dataset to use for validation.\n",
        "- `budget_milli_node_hours`: (optional) Maximum training time specified in unit of millihours (1000 = hour).\n",
        "- `disable_early_stopping`: If `True`, the entire budget is used. Else, training maybe completed before using the entire budget if the service believes it cannot further improve on the model objective measurements.\n",
        "\n",
        "The `run` method when completed returns the model resource.\n",
        "\n",
        "The execution of the training pipeline will take upto 60 minutes."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "run_automl_pipeline:image"
      },
      "outputs": [],
      "source": [
        "model = dag.run(\n",
        "    dataset=dataset,\n",
        "    model_display_name=\"salads\",\n",
        "    training_fraction_split=0.8,\n",
        "    validation_fraction_split=0.1,\n",
        "    test_fraction_split=0.1,\n",
        "    budget_milli_node_hours=20000,\n",
        "    disable_early_stopping=False,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "evaluate_the_model:mbsdk"
      },
      "source": [
        "## Review model evaluation scores\n",
        "\n",
        "After your model training has finished, you can review the evaluation scores for it using the `list_model_evaluations()` method. This method will return an iterator for each evaluation slice."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "evaluate_the_model:mbsdk"
      },
      "outputs": [],
      "source": [
        "model_evaluations = model.list_model_evaluations()\n",
        "\n",
        "for model_evaluation in model_evaluations:\n",
        "    print(model_evaluation.to_dict())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "export_model:mbsdk,image"
      },
      "source": [
        "## Export as Edge model\n",
        "\n",
        "You can export an AutoML image object detection model as a edge model which you can then custom deploy to an edge device or download locally. Use the method `export_model()` to export the model to Cloud Storage, which takes the following parameters:\n",
        "\n",
        "- `artifact_destination`: The Cloud Storage location to store the SavedFormat model artifacts to.\n",
        "- `export_format_id`: The format to save the model format as. For AutoML image object detection there is just one option:\n",
        "   - `tf-saved-model`: TensorFlow SavedFormat for deployment to a container.\n",
        "   - `tflite`: TensorFlow Lite for deployment to an edge or mobile device.\n",
        "   - `edgetpu-tflite`: TensorFlow Lite for TPU\n",
        "   - `tf-js`: TensorFlow for web client\n",
        "   - `coral-ml`: for Coral devices\n",
        "\n",
        "- `sync`: Whether to perform operational sychronously or asynchronously."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "export_model:mbsdk,image"
      },
      "outputs": [],
      "source": [
        "response = model.export_model(\n",
        "    artifact_destination=BUCKET_URI, export_format_id=\"tflite\", sync=True\n",
        ")\n",
        "\n",
        "model_package = response[\"artifactOutputUri\"]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "download_model_artifacts:tflite"
      },
      "source": [
        "#### Download the TFLite model artifacts\n",
        "\n",
        "Now that you have an exported TFLite version of your model, you can test the exported model locally, but first downloading it from Cloud Storage."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "download_model_artifacts:tflite"
      },
      "outputs": [],
      "source": [
        "! gsutil ls $model_package\n",
        "# Download the model artifacts\n",
        "! gsutil cp -r $model_package tflite\n",
        "\n",
        "tflite_path = \"tflite/model.tflite\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "instantiate_tflite_interpreter"
      },
      "source": [
        "#### Instantiate a TFLite interpreter\n",
        "\n",
        "The TFLite version of the model is not a TensorFlow SavedModel format. You cannot directly use methods like predict(). Instead, one uses the TFLite interpreter. You must first setup the interpreter for the TFLite model as follows:\n",
        "\n",
        "- Instantiate an TFLite interpreter for the TFLite model.\n",
        "- Instruct the interpreter to allocate input and output tensors for the model.\n",
        "- Get detail information about the models input and output tensors that will need to be known for prediction."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "instantiate_tflite_interpreter"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "\n",
        "interpreter = tf.lite.Interpreter(model_path=tflite_path)\n",
        "interpreter.allocate_tensors()\n",
        "\n",
        "input_details = interpreter.get_input_details()\n",
        "output_details = interpreter.get_output_details()\n",
        "input_shape = input_details[0][\"shape\"]\n",
        "\n",
        "print(\"input tensor shape\", input_shape)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "get_test_item"
      },
      "source": [
        "### Get test item\n",
        "\n",
        "You will use an arbitrary example out of the dataset as a test item. Don't be concerned that the example was likely used in training the model -- we just want to demonstrate how to make a prediction."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "get_test_item:image,224x224"
      },
      "outputs": [],
      "source": [
        "test_items = ! gsutil cat $IMPORT_FILE | head -n1\n",
        "test_item = test_items[0].split(\",\")[0]\n",
        "\n",
        "with tf.io.gfile.GFile(test_item, \"rb\") as f:\n",
        "    content = f.read()\n",
        "test_image = tf.io.decode_jpeg(content)\n",
        "print(\"test image shape\", test_image.shape)\n",
        "\n",
        "test_image = tf.image.resize(test_image, (192, 192))\n",
        "print(\"test image shape\", test_image.shape, test_image.dtype)\n",
        "\n",
        "test_image = tf.cast(test_image, dtype=tf.uint8).numpy()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "invoke_tflite_interpreter"
      },
      "source": [
        "#### Make a prediction with TFLite model\n",
        "\n",
        "Finally, you do a prediction using your TFLite model, as follows:\n",
        "\n",
        "- Convert the test image into a batch of a single image (`np.expand_dims`)\n",
        "- Set the input tensor for the interpreter to your batch of a single image (`data`).\n",
        "- Invoke the interpreter.\n",
        "- Retrieve the softmax probabilities for the prediction (`get_tensor`).\n",
        "- Determine which label had the highest probability (`np.argmax`)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "invoke_tflite_interpreter"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "\n",
        "data = np.expand_dims(test_image, axis=0)\n",
        "\n",
        "interpreter.set_tensor(input_details[0][\"index\"], data)\n",
        "\n",
        "interpreter.invoke()\n",
        "\n",
        "softmax = interpreter.get_tensor(output_details[0][\"index\"])\n",
        "\n",
        "label = np.argmax(softmax)\n",
        "\n",
        "print(label)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cleanup:mbsdk"
      },
      "source": [
        "# Cleaning up\n",
        "\n",
        "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
        "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
        "\n",
        "Otherwise, you can delete the individual resources you created in this tutorial."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cleanup:mbsdk"
      },
      "outputs": [],
      "source": [
        "delete_bucket = False\n",
        "\n",
        "# Delete the dataset using the Vertex dataset object\n",
        "dataset.delete()\n",
        "\n",
        "# Delete the model using the Vertex model object\n",
        "model.delete()\n",
        "\n",
        "# Delete the AutoML trainig job\n",
        "dag.delete()\n",
        "\n",
        "if delete_bucket:\n",
        "    ! gsutil rm -r $BUCKET_URI"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "automl_image_object_detection_export_edge.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}