This short tutorial shows how to convert a TensorFlow MobileNetV3 image classification model to OpenVINO Intermediate Representation (OpenVINO IR) format, using Model Optimizer. After creating the OpenVINO IR, load the model in OpenVINO Runtime and do inference with a sample image.
import time
from pathlib import Path
import cv2
import matplotlib.pyplot as plt
import numpy as np
import tensorflow as tf
from IPython.display import Markdown
from openvino.runtime import Core
# The paths of the source and converted models.
model_dir = Path("model")
model_dir.mkdir(exist_ok=True)
model_path = Path("model/v3-small_224_1.0_float")
ir_path = Path("model/v3-small_224_1.0_float.xml")
Load model using tf.keras.applications api and save it to the disk.
model = tf.keras.applications.MobileNetV3Small()
model.save(model_path)
Use Model Optimizer to convert a TensorFlow model to OpenVINO IR with FP16 precision. The models are saved to the current directory. Add mean values to the model and scale the output with the standard deviation with --scale_values. With these options, it is not necessary to normalize input data before propagating it through the network. The original model expects input images in RGB format. The converted model also expects images in RGB format. If you want the converted model to work with BGR images, use the --reverse-input-channels option. For more information about Model Optimizer, including a description of the command-line options, see the Model Optimizer Developer Guide. For information about the model, including input shape, expected color order and mean values, refer to the model documentation.
First construct the command for Model Optimizer, and then execute this command in the notebook by prepending the command with an !. There may be some errors or warnings in the output. When model optimization is successful, the last lines of the output will include [ SUCCESS ] Generated IR version 11 model.
# Construct the command for Model Optimizer.
mo_command = f"""mo
--saved_model_dir "{model_path}"
--input_shape "[1,224,224,3]"
--mean_values="[127.5,127.5,127.5]"
--scale_values="[127.5]"
--data_type FP16
--model_name "{model_path.name}"
--output_dir "{model_path.parent}"
"""
mo_command = " ".join(mo_command.split())
print("Model Optimizer command to convert TensorFlow to OpenVINO:")
display(Markdown(f"`{mo_command}`"))
# Run Model Optimizer if the IR model file does not exist
if not ir_path.exists():
print("Exporting TensorFlow model to IR... This may take a few minutes.")
! $mo_command
else:
print(f"IR model {ir_path} already exists.")
ie = Core()
model = ie.read_model(ir_path)
compiled_model = ie.compile_model(model=model, device_name="CPU")
input_key = compiled_model.input(0)
output_key = compiled_model.output(0)
network_input_shape = input_key.shape
Load an image, resize it, and convert it to the input shape of the network.
# The MobileNet network expects images in RGB format.
image = cv2.cvtColor(cv2.imread(filename="data/coco.jpg"), code=cv2.COLOR_BGR2RGB)
# Resize the image to the network input shape.
resized_image = cv2.resize(src=image, dsize=(224, 224))
# Transpose the image to the network input shape.
input_image = np.expand_dims(resized_image, 0)
plt.imshow(image);
result = compiled_model(input_image)[output_key]
result_index = np.argmax(result)
# Convert the inference result to a class name.
imagenet_classes = open("utils/imagenet_2012.txt").read().splitlines()
# The model description states that for this model, class 0 is background.
# Therefore, add background at the beginning of imagenet_classes
imagenet_classes = ['background'] + imagenet_classes
imagenet_classes[result_index]
Measure the time it takes to do inference on thousand images. This gives an indication of performance. For more accurate benchmarking, use the Benchmark Tool in OpenVINO. Note that many optimizations are possible to improve the performance.
num_images = 1000
start = time.perf_counter()
for _ in range(num_images):
compiled_model([input_image])
end = time.perf_counter()
time_ir = end - start
print(
f"IR model in OpenVINO Runtime/CPU: {time_ir/num_images:.4f} "
f"seconds per image, FPS: {num_images/time_ir:.2f}"
)