Skip to content

Latest commit

 

History

History
97 lines (79 loc) · 3.74 KB

README.md

File metadata and controls

97 lines (79 loc) · 3.74 KB

MobileNet V1 BFloat16 inference

Description

This document has instructions for running MobileNet V1 BFloat16 inference using Intel-optimized TensorFlow.

Datasets

Download and preprocess the ImageNet dataset using the instructions here. After running the conversion script you should have a directory with the ImageNet dataset in the TF records format.

Set the DATASET_DIR to point to the TF records directory when running .

Quick Start Scripts

Script name Description
inference_realtime.sh Runs multi instance realtime inference using 4 cores per instance for the specified precision (fp32, int8 or bfloat16) with 1000 steps, 500 warmup steps and batch-size=1. If no DATASET_DIR is set, synthetic data is used. Waits for all instances to complete, then prints a summarized throughput value.
inference_throughput.sh Runs multi instance batch inference using 1 instance per socket for the specified precision (fp32, int8 or bfloat16) with 1000 steps, 500 warmup steps and batch-size=448. If no DATASET_DIR is set, synthetic data is used. Waits for all instances to complete, then prints a summarized throughput value.
accuracy.sh Measures the inference accuracy (providing a DATASET_DIR environment variable is required) for the specified precision (fp32, int8 or bfloat16) with batch-size=100.

Run the model

Setup your environment using the instructions below, depending on if you are using AI Kit:

Setup using AI Kit on Linux Setup without AI Kit on Linux

To run using AI Kit on Linux you will need:

  • numactl
  • wget
  • Activate the tensorflow conda environment 
    conda activate tensorflow

To run without AI Kit on Linux you will need:

  • Python 3
  • intel-tensorflow>=2.5.0
  • git
  • numactl
  • wget
  • A clone of the Model Zoo repo
    git clone https://github.com/IntelAI/models.git

After finishing the setup above, download the pretrained model and set the PRETRAINED_MODEL environment var to the path to the frozen graph.

wget https://storage.googleapis.com/intel-optimized-tensorflow/models/v1_8/mobilenet_v1_1.0_224_frozen.pb
export PRETRAINED_MODEL=$(pwd)/mobilenet_v1_1.0_224_frozen.pb

Set environment variables for the path to your DATASET_DIR for ImageNet and an OUTPUT_DIR where log files will be written. Navigate to your model zoo directory and then run a quickstart script.

Run on Linux:

# cd to your model zoo directory
cd models

# Set environment variables
export PRETRAINED_MODEL=<path to the frozen graph downloaded above>
export DATASET_DIR=<path to the ImageNet TF records>
export OUTPUT_DIR=<path to the directory where log files will be written>
export PRECISION=bfloat16
# For a custom batch size, set env var `BATCH_SIZE` or it will run with a default value.
export BATCH_SIZE=<customized batch size value>

# Run a quickstart script (for example, inference realtime.sh)
./quickstart/image_recognition/tensorflow/mobilenet_v1/inference/cpu/<script name>.sh

Additional Resources

To run more advanced use cases, see the instructions here for calling the launch_benchmark.py script directly.