Notebook

Training Hello world model for Microcontrollers¶

In this post, we'll cover the basic tutorial for training simple regression model with tensorflow lite for for Microcontrollers(TFLM). This post is the summary of youtube video "TinyML Book Screencast - Training the Hello World model", presented by peter warden.

toc: true
badges: true
comments: true
author: Chanseok Kang
categories: [Python, Deep_Learning, Tensorflow-Keras, Tensorflow-Lite]
image:

In [1]:

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

plt.rcParams['figure.figsize'] = (16, 10)
plt.rcParams['text.usetex'] = True
plt.rc('font', size=15)

Tensorflow Lite for Microcontrollers¶

TensorFlow Lite for Microcontrollers (TFLM for short) is designed to run machine learning models on microcontrollers and other devices with only few kilobytes of memory. For the purpose of deploying machine learning model on embedded devices, it is also called TinyML. As you know already, Tensorflow is one of commonly-used deep learning frameworks, and from version 2.x they offer some machine learning features for embedded systems via tensorflow lite. Unlike high performance mobile processor(like cortex-A series), Microcontroller (Cortex-M series or ESP32) has low power consumptions and it can deploy in various ways of customer products, like refrigerator, wash-machine and so on.

Google introduced several supported boards for test,

Arduino Nano 33 BLE Sense (using Arduino IDE)
SparkFun Edge (building directly from source)
STM32F746 Discovery kit (using Mbed)
Adafruit EdgeBadge (using Arduino IDE)
Adafruit TensorFlow Lite for Microcontrollers Kit (using Arduino IDE)
Adafruit Circuit Playground Bluefruit (using Arduino IDE)
Espressif ESP32-DevKitC (using ESP IDF)
Espressif ESP-EYE (using ESP IDF)

Here, we'll implement the simple regression model in Sparkfun Edge. Most of contents are covered in Pete Warden's screencast. More informations are included in his book.

youtube: https://www.youtube.com/watch?v=8N6-WQsxwxA&t=205s

Prerequisites for implementation¶

Before beginning, it requires some hardwares and basic knowledges,

hardware
- Sparkfun edge
- CH340c USB-c to UART converter
- Desktop PC for training model (or you can use google colab)
Knowledge
- Python
- C/C++
- Basic knowledge related on OS

Hello world¶

Actually, "Hello world" may be the first program we faced, since it can show simple interaction between human and computer. TinyML also has simple example of "Hello world". Instead of Printing, we will build a model to generate the sine wave. So maybe our hypothesis will be like this,

$$ \tilde{H(x)} = \sin(x) $$

At first, we load the required packages,

In [7]:

import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt
import math

plt.rcParams['figure.figsize'] = (16, 10)
plt.rc('font', size=15)

# define random seed for reproducibility
np.random.seed(1) # numpy seed
tf.random.set_seed(1) # tensorflow global random seed

In [8]:

print('Numpy: {}'.format(np.__version__))
print('Tensorflow: {}'.format(tf.__version__))

Numpy: 1.18.1
Tensorflow: 2.2.0

To train the model, it requires a sort of data, namely training data. In our case, we will sample the random data from numpy and generate the actual output from known model(the sine function)

In [14]:

# Generate a uniformly distributed set of random numbers in the range from
# 0 to 2π, which covers a complete sine wave oscillation
X = np.random.uniform(
    low=0, high=2*math.pi, size=10000).astype(np.float32)

# Shuffle the values to guarantee they're not in order
np.random.shuffle(X)

# Calculate the corresponding sine values
y = np.sin(X).astype(np.float32)

# Plot our data. The 'b.' argument tells the library to print blue dots.
plt.plot(X, y, 'b.')
plt.grid()
plt.show()

But this data doesn't reflect the real-world data, because there is no variation of distribution in dataset, also known as noise. We can add random data for each x, so it makes to seem more randomly distributed.

In [15]:

# Add a small random number to each y value
y += 0.1 * np.random.randn(*y.shape)

# Plot our data
plt.plot(X, y, 'b.')
plt.grid()
plt.show()

Through this, we expect that the model is approximated with sinusoid curve if it is well trained.

To use it for training, we are going to preprocess the data. We'll split it with following proportions,

Train data: 60%
Validation data: 20%
Test data: 20%

In [18]:

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=1)
X_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.25, random_state=1)

# Plot the data in each partition in different colors:
plt.plot(X_train, y_train, 'b.', label="Train")
plt.plot(X_val, y_val, 'y.', label="Validate")
plt.plot(X_test, y_test, 'r.', label="Test")
plt.legend()
plt.grid()
plt.show()

To train the model, we will use sequential model in tensorflow-keras, and add two Dense layer. Then we will add adam optimizer, and set mean squared error for loss.

In [20]:

model = tf.keras.Sequential(name='sine')
model.add(tf.keras.layers.Dense(16, activation='relu', input_shape=(1, )))
model.add(tf.keras.layers.Dense(1))

model.summary()

Model: "sine"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_2 (Dense)              (None, 16)                32        
_________________________________________________________________
dense_3 (Dense)              (None, 1)                 17        
=================================================================
Total params: 49
Trainable params: 49
Non-trainable params: 0
_________________________________________________________________

In [22]:

model.compile(optimizer='adam', loss='mse', metrics=['accuracy', 'mae'])

Now, it's time to train the model.

In [24]:

history = model.fit(X_train, y_train, epochs=1000, batch_size=15, validation_data=(X_val, y_val), verbose=False)

In [25]:

# Draw a graph of the loss, which is the distance between
# the predicted and actual values during training and validation.
loss = history.history['loss']
val_loss = history.history['val_loss']

epochs = range(1, len(loss) + 1)

plt.plot(epochs, loss, 'g.', label='Training loss')
plt.plot(epochs, val_loss, 'b', label='Validation loss')
plt.title('Training and validation loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend()
plt.show()

In [26]:

# Exclude the first few epochs so the graph is easier to read
SKIP = 50

plt.plot(epochs[SKIP:], loss[SKIP:], 'g.', label='Training loss')
plt.plot(epochs[SKIP:], val_loss[SKIP:], 'b.', label='Validation loss')
plt.title('Training and validation loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend()
plt.show()

In [27]:

# Calculate and print the loss on our test dataset
loss = model.evaluate(X_test, y_test)

# Make predictions based on our test dataset
predictions = model.predict(X_test)

# Graph the predictions against the actual values
plt.clf()
plt.title('Comparison of predictions and actual values')
plt.plot(X_test, y_test, 'b.', label='Actual')
plt.plot(X_test, predictions, 'r.', label='Predicted')
plt.legend()
plt.show()

63/63 [==============================] - 0s 1ms/step - loss: 0.0135 - accuracy: 0.0000e+00 - mae: 0.0921

In [30]:

model_2 = tf.keras.Sequential(name='sine2')

# First layer takes a scalar input and feeds it through 16 "neurons". The
# neurons decide whether to activate based on the 'relu' activation function.
model_2.add(tf.keras.layers.Dense(16, activation='relu', input_shape=(1,)))

# The new second layer may help the network learn more complex representations
model_2.add(tf.keras.layers.Dense(16, activation='relu'))

# Final layer is a single neuron, since we want to output a single value
model_2.add(tf.keras.layers.Dense(1))

# Compile the model using a standard optimizer and loss function for regression
model_2.compile(optimizer='adam', loss='mse', metrics=['accuracy', 'mae'])

model_2.summary()

Model: "sine2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_7 (Dense)              (None, 16)                32        
_________________________________________________________________
dense_8 (Dense)              (None, 16)                272       
_________________________________________________________________
dense_9 (Dense)              (None, 1)                 17        
=================================================================
Total params: 321
Trainable params: 321
Non-trainable params: 0
_________________________________________________________________

In [31]:

history = model_2.fit(X_train, y_train, epochs=500, batch_size=64,
                    validation_data=(X_val, y_val), verbose=False)

In [34]:

# Draw a graph of the loss, which is the distance between
# the predicted and actual values during training and validation.
loss = history.history['loss']
val_loss = history.history['val_loss']

epochs = range(1, len(loss) + 1)

# Exclude the first few epochs so the graph is easier to read
SKIP = 100

plt.figure()
plt.subplot(1, 2, 1)

plt.plot(epochs[SKIP:], loss[SKIP:], 'g.', label='Training loss')
plt.plot(epochs[SKIP:], val_loss[SKIP:], 'b.', label='Validation loss')
plt.title('Training and validation loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend()

plt.subplot(1, 2, 2)

# Draw a graph of mean absolute error, which is another way of
# measuring the amount of error in the prediction.
mae = history.history['mae']
val_mae = history.history['val_mae']

plt.plot(epochs[SKIP:], mae[SKIP:], 'g.', label='Training MAE')
plt.plot(epochs[SKIP:], val_mae[SKIP:], 'b.', label='Validation MAE')
plt.title('Training and validation mean absolute error')
plt.xlabel('Epochs')
plt.ylabel('MAE')
plt.legend()

plt.tight_layout()

In [36]:

# Calculate and print the loss on our test dataset
loss = model_2.evaluate(X_test, y_test)

# Make predictions based on our test dataset
predictions = model_2.predict(X_test)

# Graph the predictions against the actual values
plt.clf()
plt.title('Comparison of predictions and actual values')
plt.plot(X_test, y_test, 'b.', label='Actual')
plt.plot(X_test, predictions, 'r.', label='Predicted')
plt.legend()
plt.show()

63/63 [==============================] - 0s 972us/step - loss: 0.0111 - accuracy: 0.0000e+00 - mae: 0.0840

In [37]:

import os
MODEL_DIR = './models/'
if not os.path.exists(MODEL_DIR):
    os.mkdir(MODEL_DIR)

In [45]:

# Convert the model to the TensorFlow Lite format without quantization
converter = tf.lite.TFLiteConverter.from_keras_model(model_2)
model_no_quant_tflite = converter.convert()

# # Save the model to disk
open(MODEL_DIR + 'model_no_quant.tflite', "wb").write(model_no_quant_tflite)

# Convert the model to the TensorFlow Lite format with quantization
def representative_dataset():
    for i in range(500):
        yield([X_train[i].reshape(1, 1)])
# Set the optimization flag.
converter.optimizations = [tf.lite.Optimize.DEFAULT]
# Enforce full-int8 quantization (except inputs/outputs which are always float)
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
# Provide a representative dataset to ensure we quantize correctly.
converter.representative_dataset = representative_dataset
model_tflite = converter.convert()

# Save the model to disk
open(MODEL_DIR + 'model.tflite', "wb").write(model_tflite)

Out[45]:

In [46]:

model_no_quant_size = os.path.getsize(MODEL_DIR + 'model_no_quant.tflite')
print("Model is %d bytes" % model_no_quant_size)
model_size = os.path.getsize(MODEL_DIR + 'model.tflite')
print("Quantized model is %d bytes" % model_size)
difference = model_no_quant_size - model_size
print("Difference is %d bytes" % difference)

Model is 2656 bytes
Quantized model is 2672 bytes
Difference is -16 bytes

In [49]:

# Instantiate an interpreter for each model
model_no_quant = tf.lite.Interpreter(MODEL_DIR + 'model_no_quant.tflite')
model = tf.lite.Interpreter(MODEL_DIR + 'model.tflite')

# Allocate memory for each model
model_no_quant.allocate_tensors()
model.allocate_tensors()

# Get the input and output tensors so we can feed in values and get the results
model_no_quant_input = model_no_quant.tensor(model_no_quant.get_input_details()[0]["index"])
model_no_quant_output = model_no_quant.tensor(model_no_quant.get_output_details()[0]["index"])
model_input = model.tensor(model.get_input_details()[0]["index"])
model_output = model.tensor(model.get_output_details()[0]["index"])

# Create arrays to store the results
model_no_quant_predictions = np.empty(X_test.size)
model_predictions = np.empty(X_test.size)

# Run each model's interpreter for each value and store the results in arrays
for i in range(X_test.size):
    model_no_quant_input().fill(X_test[i])
    model_no_quant.invoke()
    model_no_quant_predictions[i] = model_no_quant_output()[0]

    model_input().fill(X_test[i])
    model.invoke()
    model_predictions[i] = model_output()[0]

# See how they line up with the data
plt.clf()
plt.title('Comparison of various models against actual values')
plt.plot(X_test, y_test, 'bo', label='Actual values', alpha=0.4)
plt.plot(X_test, predictions, 'ro', label='Original predictions')
plt.plot(X_test, model_no_quant_predictions, 'bx', label='Lite predictions')
plt.plot(X_test, model_predictions, 'gx', label='Lite quantized predictions')
plt.legend()
plt.show()

In [53]:

!xxd -i {MODEL_DIR + 'model.tflite'} > {MODEL_DIR + 'model.cc'}

In [56]:

!cat {MODEL_DIR + 'model.cc'}

unsigned char __models_model_tflite[] = {
  0x20, 0x00, 0x00, 0x00, 0x54, 0x46, 0x4c, 0x33, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x12, 0x00, 0x1c, 0x00, 0x04, 0x00, 0x08, 0x00, 0x0c, 0x00,
  0x10, 0x00, 0x14, 0x00, 0x00, 0x00, 0x18, 0x00, 0x12, 0x00, 0x00, 0x00,
  0x03, 0x00, 0x00, 0x00, 0xfc, 0x09, 0x00, 0x00, 0x14, 0x03, 0x00, 0x00,
  0xfc, 0x02, 0x00, 0x00, 0x3c, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00,
  0x01, 0x00, 0x00, 0x00, 0x0c, 0x00, 0x00, 0x00, 0x08, 0x00, 0x0c, 0x00,
  0x04, 0x00, 0x08, 0x00, 0x08, 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00,
  0x0b, 0x00, 0x00, 0x00, 0x13, 0x00, 0x00, 0x00, 0x6d, 0x69, 0x6e, 0x5f,
  0x72, 0x75, 0x6e, 0x74, 0x69, 0x6d, 0x65, 0x5f, 0x76, 0x65, 0x72, 0x73,
  0x69, 0x6f, 0x6e, 0x00, 0x0c, 0x00, 0x00, 0x00, 0xa8, 0x02, 0x00, 0x00,
  0x94, 0x02, 0x00, 0x00, 0x38, 0x02, 0x00, 0x00, 0xd4, 0x01, 0x00, 0x00,
  0xc0, 0x01, 0x00, 0x00, 0x9c, 0x01, 0x00, 0x00, 0x88, 0x00, 0x00, 0x00,
  0x64, 0x00, 0x00, 0x00, 0x58, 0x00, 0x00, 0x00, 0x44, 0x00, 0x00, 0x00,
  0x30, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0xf6, 0xfd, 0xff, 0xff,
  0x04, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00, 0x31, 0x2e, 0x35, 0x2e,
  0x30, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x04, 0xfd, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x14, 0xfd, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x24, 0xfd, 0xff, 0xff,
  0x00, 0x00, 0x00, 0x00, 0x46, 0xfe, 0xff, 0xff, 0x04, 0x00, 0x00, 0x00,
  0x10, 0x00, 0x00, 0x00, 0xbb, 0x27, 0xf5, 0xf6, 0x04, 0xe3, 0x25, 0xe5,
  0x7f, 0x69, 0x18, 0xec, 0xdc, 0x32, 0xec, 0xe4, 0x00, 0x00, 0x00, 0x00,
  0x66, 0xfe, 0xff, 0xff, 0x04, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00,
  0x01, 0x17, 0x07, 0x15, 0xf2, 0x10, 0xef, 0xfd, 0xdd, 0xf7, 0xf5, 0x08,
  0x29, 0xbe, 0x29, 0xe5, 0xef, 0x05, 0xf7, 0xea, 0x0e, 0x14, 0x11, 0xf9,
  0xfc, 0x10, 0x0a, 0xeb, 0xee, 0x04, 0xe8, 0x00, 0x0a, 0x04, 0xf4, 0x10,
  0xf6, 0xf0, 0xf0, 0xe7, 0xef, 0x01, 0xff, 0x06, 0xf8, 0x00, 0x18, 0x00,
  0xe8, 0x06, 0x10, 0xfb, 0x0f, 0x17, 0xe8, 0xf1, 0x0d, 0x16, 0xfe, 0x07,
  0x00, 0xfd, 0xea, 0x14, 0x05, 0xf5, 0xf5, 0x0a, 0x06, 0xeb, 0x18, 0x11,
  0xea, 0x0e, 0xeb, 0x0a, 0xee, 0x1a, 0x0c, 0x0c, 0xf4, 0xe8, 0xee, 0x01,
  0xf9, 0xfe, 0x17, 0xf8, 0xf2, 0xf4, 0x02, 0x0b, 0x0d, 0x01, 0xe8, 0x05,
  0x10, 0x14, 0xf8, 0xf9, 0x16, 0x07, 0xee, 0xfa, 0xfa, 0x01, 0x0e, 0x08,
  0xfc, 0xfe, 0xfa, 0x06, 0x0a, 0x01, 0xea, 0x0c, 0xef, 0x0e, 0xeb, 0xf2,
  0x10, 0x14, 0x02, 0x04, 0xf3, 0xe6, 0xe9, 0xef, 0xf9, 0x17, 0xfe, 0xee,
  0xfe, 0x06, 0x0a, 0x0d, 0xe0, 0xf1, 0xe7, 0x06, 0x1d, 0xa3, 0x2a, 0xf9,
  0x0e, 0x18, 0xf9, 0xfd, 0xef, 0x08, 0x16, 0x09, 0xe9, 0x02, 0xd6, 0xfe,
  0xfd, 0x81, 0x1f, 0x18, 0x02, 0xf8, 0xec, 0xeb, 0x01, 0xed, 0xec, 0x14,
  0x21, 0xfa, 0x00, 0x08, 0x05, 0x1d, 0x0d, 0xf8, 0xea, 0xfd, 0x0c, 0x15,
  0x0b, 0x14, 0xf7, 0xf8, 0x12, 0x05, 0xf1, 0xf6, 0x00, 0xf2, 0x04, 0x11,
  0xf5, 0xf9, 0xf8, 0xf6, 0xee, 0xf4, 0xf5, 0x0f, 0xf1, 0xfe, 0x10, 0x11,
  0x0c, 0xf3, 0xe7, 0xfc, 0xef, 0x1b, 0xfb, 0x0b, 0xfc, 0xe7, 0x10, 0x12,
  0x06, 0xf8, 0x15, 0xf2, 0x05, 0x00, 0xea, 0xf7, 0x04, 0xef, 0xfd, 0x14,
  0x19, 0x0c, 0x12, 0x16, 0x0b, 0x0e, 0x0f, 0x0a, 0xfd, 0xeb, 0x05, 0x0c,
  0x09, 0x0b, 0x0b, 0x16, 0x0f, 0xf4, 0xf4, 0x18, 0x0e, 0x19, 0xff, 0xeb,
  0x1d, 0xea, 0x1e, 0xfa, 0x00, 0x00, 0x00, 0x00, 0x76, 0xff, 0xff, 0xff,
  0x04, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00, 0xcc, 0x7f, 0xee, 0xdc,
  0xc4, 0xfd, 0xc1, 0xd0, 0x65, 0x9e, 0x6b, 0x0c, 0x07, 0x30, 0x1a, 0x47,
  0x00, 0x00, 0x00, 0x00, 0x96, 0xff, 0xff, 0xff, 0x04, 0x00, 0x00, 0x00,
  0x04, 0x00, 0x00, 0x00, 0xee, 0xf0, 0xff, 0xff, 0xa6, 0xff, 0xff, 0xff,
  0x04, 0x00, 0x00, 0x00, 0x40, 0x00, 0x00, 0x00, 0x06, 0x12, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0xd9, 0x01, 0x00, 0x00, 0x38, 0x01, 0x00, 0x00,
  0x2d, 0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00, 0x82, 0xfc, 0xff, 0xff,
  0xb8, 0xff, 0xff, 0xff, 0x9c, 0x06, 0x00, 0x00, 0xf6, 0x03, 0x00, 0x00,
  0x0f, 0xfa, 0xff, 0xff, 0x5e, 0x02, 0x00, 0x00, 0x83, 0xff, 0xff, 0xff,
  0x4b, 0xfa, 0xff, 0xff, 0x7f, 0x02, 0x00, 0x00, 0x69, 0x07, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x06, 0x00, 0x08, 0x00, 0x04, 0x00, 0x06, 0x00, 0x00, 0x00,
  0x04, 0x00, 0x00, 0x00, 0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x86, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x5d, 0xf1, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00,
  0x8c, 0xe5, 0xff, 0xff, 0xdd, 0xff, 0xff, 0xff, 0xca, 0x17, 0x00, 0x00,
  0xde, 0xee, 0xff, 0xff, 0xa5, 0x18, 0x00, 0x00, 0xa3, 0xff, 0xff, 0xff,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x44, 0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x54, 0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0f, 0x00, 0x00, 0x00,
  0x4d, 0x4c, 0x49, 0x52, 0x20, 0x43, 0x6f, 0x6e, 0x76, 0x65, 0x72, 0x74,
  0x65, 0x64, 0x2e, 0x00, 0x01, 0x00, 0x00, 0x00, 0x14, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x0e, 0x00, 0x18, 0x00, 0x04, 0x00, 0x08, 0x00, 0x0c, 0x00,
  0x10, 0x00, 0x14, 0x00, 0x0e, 0x00, 0x00, 0x00, 0x58, 0x01, 0x00, 0x00,
  0x4c, 0x01, 0x00, 0x00, 0x40, 0x01, 0x00, 0x00, 0x14, 0x00, 0x00, 0x00,
  0x04, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x6d, 0x61, 0x69, 0x6e,
  0x00, 0x00, 0x00, 0x00, 0x05, 0x00, 0x00, 0x00, 0x04, 0x01, 0x00, 0x00,
  0xb8, 0x00, 0x00, 0x00, 0x70, 0x00, 0x00, 0x00, 0x28, 0x00, 0x00, 0x00,
  0x04, 0x00, 0x00, 0x00, 0x1a, 0xff, 0xff, 0xff, 0x02, 0x00, 0x00, 0x00,
  0x10, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00,
  0x0b, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x09, 0x00, 0x00, 0x00,
  0xca, 0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x08, 0x1c, 0x00, 0x00, 0x00,
  0x10, 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x04, 0x00, 0x04, 0x00,
  0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x09, 0x00, 0x00, 0x00,
  0x03, 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x06, 0x00, 0x00, 0x00,
  0x03, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0e, 0x00, 0x14, 0x00, 0x00, 0x00,
  0x08, 0x00, 0x0c, 0x00, 0x07, 0x00, 0x10, 0x00, 0x0e, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x08, 0x1c, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00,
  0x04, 0x00, 0x00, 0x00, 0xba, 0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x01,
  0x01, 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00,
  0x07, 0x00, 0x00, 0x00, 0x05, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x0e, 0x00, 0x16, 0x00, 0x00, 0x00, 0x08, 0x00, 0x0c, 0x00,
  0x07, 0x00, 0x10, 0x00, 0x0e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08,
  0x24, 0x00, 0x00, 0x00, 0x18, 0x00, 0x00, 0x00, 0x0c, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x06, 0x00, 0x08, 0x00, 0x07, 0x00, 0x06, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x01, 0x01, 0x00, 0x00, 0x00, 0x07, 0x00, 0x00, 0x00,
  0x03, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00,
  0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0a, 0x00, 0x10, 0x00, 0x04, 0x00,
  0x08, 0x00, 0x0c, 0x00, 0x0a, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00,
  0x10, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x0a, 0x00, 0x00, 0x00,
  0x01, 0x00, 0x00, 0x00, 0x0b, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00,
  0x0a, 0x00, 0x00, 0x00, 0x0c, 0x00, 0x00, 0x00, 0xec, 0x04, 0x00, 0x00,
  0x60, 0x04, 0x00, 0x00, 0xf0, 0x03, 0x00, 0x00, 0x7c, 0x03, 0x00, 0x00,
  0x00, 0x03, 0x00, 0x00, 0x84, 0x02, 0x00, 0x00, 0x08, 0x02, 0x00, 0x00,
  0x74, 0x01, 0x00, 0x00, 0xe0, 0x00, 0x00, 0x00, 0x6c, 0x00, 0x00, 0x00,
  0x3c, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0xd8, 0xff, 0xff, 0xff,
  0x18, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00,
  0x49, 0x64, 0x65, 0x6e, 0x74, 0x69, 0x74, 0x79, 0x00, 0x00, 0x00, 0x00,
  0x02, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00,
  0x0c, 0x00, 0x0c, 0x00, 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, 0x00,
  0x0c, 0x00, 0x00, 0x00, 0x1c, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00,
  0x0d, 0x00, 0x00, 0x00, 0x64, 0x65, 0x6e, 0x73, 0x65, 0x5f, 0x37, 0x5f,
  0x69, 0x6e, 0x70, 0x75, 0x74, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00,
  0x01, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0xb2, 0xfb, 0xff, 0xff,
  0x00, 0x00, 0x00, 0x09, 0x5c, 0x00, 0x00, 0x00, 0x0a, 0x00, 0x00, 0x00,
  0x40, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0xa4, 0xfb, 0xff, 0xff,
  0x2c, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x14, 0x00, 0x00, 0x00,
  0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x36, 0xcb, 0x03, 0x3c,
  0x01, 0x00, 0x00, 0x00, 0x12, 0xb2, 0x82, 0x3f, 0x01, 0x00, 0x00, 0x00,
  0xc4, 0xdc, 0x83, 0xbf, 0x0d, 0x00, 0x00, 0x00, 0x49, 0x64, 0x65, 0x6e,
  0x74, 0x69, 0x74, 0x79, 0x5f, 0x69, 0x6e, 0x74, 0x38, 0x00, 0x00, 0x00,
  0x02, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00,
  0x22, 0xfc, 0xff, 0xff, 0x00, 0x00, 0x00, 0x09, 0x7c, 0x00, 0x00, 0x00,
  0x09, 0x00, 0x00, 0x00, 0x44, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00,
  0x14, 0xfc, 0xff, 0xff, 0x30, 0x00, 0x00, 0x00, 0x24, 0x00, 0x00, 0x00,
  0x18, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00,
  0x80, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00,
  0x01, 0x00, 0x00, 0x00, 0x35, 0xfa, 0x12, 0x3c, 0x01, 0x00, 0x00, 0x00,
  0x3b, 0x67, 0x12, 0x40, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x28, 0x00, 0x00, 0x00, 0x73, 0x69, 0x6e, 0x65, 0x32, 0x2f, 0x64, 0x65,
  0x6e, 0x73, 0x65, 0x5f, 0x38, 0x2f, 0x52, 0x65, 0x6c, 0x75, 0x3b, 0x73,
  0x69, 0x6e, 0x65, 0x32, 0x2f, 0x64, 0x65, 0x6e, 0x73, 0x65, 0x5f, 0x38,
  0x2f, 0x42, 0x69, 0x61, 0x73, 0x41, 0x64, 0x64, 0x00, 0x00, 0x00, 0x00,
  0x02, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00,
  0xb2, 0xfc, 0xff, 0xff, 0x00, 0x00, 0x00, 0x09, 0x7c, 0x00, 0x00, 0x00,
  0x08, 0x00, 0x00, 0x00, 0x44, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00,
  0xa4, 0xfc, 0xff, 0xff, 0x30, 0x00, 0x00, 0x00, 0x24, 0x00, 0x00, 0x00,
  0x18, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00,
  0x80, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00,
  0x01, 0x00, 0x00, 0x00, 0xaf, 0xbc, 0x6e, 0x3c, 0x01, 0x00, 0x00, 0x00,
  0xf2, 0xcd, 0x6d, 0x40, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x28, 0x00, 0x00, 0x00, 0x73, 0x69, 0x6e, 0x65, 0x32, 0x2f, 0x64, 0x65,
  0x6e, 0x73, 0x65, 0x5f, 0x37, 0x2f, 0x52, 0x65, 0x6c, 0x75, 0x3b, 0x73,
  0x69, 0x6e, 0x65, 0x32, 0x2f, 0x64, 0x65, 0x6e, 0x73, 0x65, 0x5f, 0x37,
  0x2f, 0x42, 0x69, 0x61, 0x73, 0x41, 0x64, 0x64, 0x00, 0x00, 0x00, 0x00,
  0x02, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00,
  0x42, 0xfd, 0xff, 0xff, 0x00, 0x00, 0x00, 0x09, 0x64, 0x00, 0x00, 0x00,
  0x07, 0x00, 0x00, 0x00, 0x40, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00,
  0x34, 0xfd, 0xff, 0xff, 0x2c, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00,
  0x14, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00,
  0x66, 0xa4, 0x5d, 0x3c, 0x01, 0x00, 0x00, 0x00, 0x1d, 0xe9, 0xdb, 0x3f,
  0x01, 0x00, 0x00, 0x00, 0xd8, 0x2c, 0x6e, 0xbf, 0x14, 0x00, 0x00, 0x00,
  0x73, 0x69, 0x6e, 0x65, 0x32, 0x2f, 0x64, 0x65, 0x6e, 0x73, 0x65, 0x5f,
  0x39, 0x2f, 0x4d, 0x61, 0x74, 0x4d, 0x75, 0x6c, 0x00, 0x00, 0x00, 0x00,
  0x02, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00,
  0xba, 0xfd, 0xff, 0xff, 0x00, 0x00, 0x00, 0x09, 0x64, 0x00, 0x00, 0x00,
  0x06, 0x00, 0x00, 0x00, 0x40, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00,
  0xac, 0xfd, 0xff, 0xff, 0x2c, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00,
  0x14, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00,
  0x30, 0xc2, 0x89, 0x3c, 0x01, 0x00, 0x00, 0x00, 0xfc, 0x2b, 0x34, 0x3f,
  0x01, 0x00, 0x00, 0x00, 0xac, 0xae, 0x08, 0xc0, 0x14, 0x00, 0x00, 0x00,
  0x73, 0x69, 0x6e, 0x65, 0x32, 0x2f, 0x64, 0x65, 0x6e, 0x73, 0x65, 0x5f,
  0x38, 0x2f, 0x4d, 0x61, 0x74, 0x4d, 0x75, 0x6c, 0x00, 0x00, 0x00, 0x00,
  0x02, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00,
  0x32, 0xfe, 0xff, 0xff, 0x00, 0x00, 0x00, 0x09, 0x64, 0x00, 0x00, 0x00,
  0x05, 0x00, 0x00, 0x00, 0x40, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00,
  0x24, 0xfe, 0xff, 0xff, 0x2c, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00,
  0x14, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00,
  0x30, 0xdc, 0x98, 0x3b, 0x01, 0x00, 0x00, 0x00, 0x78, 0xaa, 0x17, 0x3f,
  0x01, 0x00, 0x00, 0x00, 0x02, 0x07, 0xea, 0xbe, 0x14, 0x00, 0x00, 0x00,
  0x73, 0x69, 0x6e, 0x65, 0x32, 0x2f, 0x64, 0x65, 0x6e, 0x73, 0x65, 0x5f,
  0x37, 0x2f, 0x4d, 0x61, 0x74, 0x4d, 0x75, 0x6c, 0x00, 0x00, 0x00, 0x00,
  0x02, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00,
  0xaa, 0xfe, 0xff, 0xff, 0x00, 0x00, 0x00, 0x02, 0x60, 0x00, 0x00, 0x00,
  0x04, 0x00, 0x00, 0x00, 0x2c, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00,
  0x24, 0xff, 0xff, 0xff, 0x18, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00,
  0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0xc5, 0x80, 0xfe, 0x38,
  0x24, 0x00, 0x00, 0x00, 0x73, 0x69, 0x6e, 0x65, 0x32, 0x2f, 0x64, 0x65,
  0x6e, 0x73, 0x65, 0x5f, 0x39, 0x2f, 0x42, 0x69, 0x61, 0x73, 0x41, 0x64,
  0x64, 0x2f, 0x52, 0x65, 0x61, 0x64, 0x56, 0x61, 0x72, 0x69, 0x61, 0x62,
  0x6c, 0x65, 0x4f, 0x70, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00,
  0x01, 0x00, 0x00, 0x00, 0x1a, 0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x02,
  0x5c, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x28, 0x00, 0x00, 0x00,
  0x04, 0x00, 0x00, 0x00, 0x94, 0xff, 0xff, 0xff, 0x14, 0x00, 0x00, 0x00,
  0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x11, 0x78, 0x80, 0x39,
  0x24, 0x00, 0x00, 0x00, 0x73, 0x69, 0x6e, 0x65, 0x32, 0x2f, 0x64, 0x65,
  0x6e, 0x73, 0x65, 0x5f, 0x38, 0x2f, 0x42, 0x69, 0x61, 0x73, 0x41, 0x64,
  0x64, 0x2f, 0x52, 0x65, 0x61, 0x64, 0x56, 0x61, 0x72, 0x69, 0x61, 0x62,
  0x6c, 0x65, 0x4f, 0x70, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00,
  0x10, 0x00, 0x00, 0x00, 0x86, 0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x02,
  0x68, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x34, 0x00, 0x00, 0x00,
  0x10, 0x00, 0x00, 0x00, 0x0c, 0x00, 0x0c, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x04, 0x00, 0x08, 0x00, 0x0c, 0x00, 0x00, 0x00, 0x14, 0x00, 0x00, 0x00,
  0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x7f, 0x9f, 0xef, 0x38,
  0x24, 0x00, 0x00, 0x00, 0x73, 0x69, 0x6e, 0x65, 0x32, 0x2f, 0x64, 0x65,
  0x6e, 0x73, 0x65, 0x5f, 0x37, 0x2f, 0x42, 0x69, 0x61, 0x73, 0x41, 0x64,
  0x64, 0x2f, 0x52, 0x65, 0x61, 0x64, 0x56, 0x61, 0x72, 0x69, 0x61, 0x62,
  0x6c, 0x65, 0x4f, 0x70, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00,
  0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0e, 0x00, 0x18, 0x00, 0x08, 0x00,
  0x07, 0x00, 0x0c, 0x00, 0x10, 0x00, 0x14, 0x00, 0x0e, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x09, 0x6c, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00,
  0x4c, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00, 0x0c, 0x00, 0x14, 0x00,
  0x04, 0x00, 0x08, 0x00, 0x0c, 0x00, 0x10, 0x00, 0x0c, 0x00, 0x00, 0x00,
  0x2c, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x14, 0x00, 0x00, 0x00,
  0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x80, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xff, 0xff, 0x01, 0x00, 0x00, 0x00, 0x04, 0xa7, 0xc8, 0x3c,
  0x01, 0x00, 0x00, 0x00, 0x5d, 0xde, 0xc7, 0x40, 0x01, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x12, 0x00, 0x00, 0x00, 0x64, 0x65, 0x6e, 0x73,
  0x65, 0x5f, 0x37, 0x5f, 0x69, 0x6e, 0x70, 0x75, 0x74, 0x5f, 0x69, 0x6e,
  0x74, 0x38, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00,
  0x01, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x3c, 0x00, 0x00, 0x00,
  0x28, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0a, 0x00,
  0x0e, 0x00, 0x07, 0x00, 0x00, 0x00, 0x08, 0x00, 0x0a, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x06, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x06, 0x00,
  0x06, 0x00, 0x05, 0x00, 0x06, 0x00, 0x00, 0x00, 0x00, 0x72, 0x0a, 0x00,
  0x0c, 0x00, 0x07, 0x00, 0x00, 0x00, 0x08, 0x00, 0x0a, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x09, 0x04, 0x00, 0x00, 0x00
};
unsigned int __models_model_tflite_len = 2672;

In [ ]: