Notebook

🚩 Create a free WhyLabs account to get more value out of whylogs!

Did you know you can store, visualize, and monitor whylogs profiles with the WhyLabs Observability Platform? Sign up for a free WhyLabs account to leverage the power of whylogs and WhyLabs together!

Integrating Whylogs into your Flask Flow¶

Now that you've gone through the basics, let's get into how to integrate whylogs into your current work flow. Today, let's look at Flask to build a web app that wil use data from IRIS and us it for a prediction. Notice that we will log both the input received and the prediction!

What you'll Need¶

Docker
pandas
scikit-learn
Flask

Overview¶

Data Flow by Felipe de Pontes Adachi

We’ll deploy locally a Flask application, which is responsible for serving the user with the requested predictions through a REST endpoint. Our application will use the whylogs library to create statistical profiles of both input and output features of our application during production. These statistical properties will then be sent in microbatches to either to a local writer or WhyLabs at fixed intervals. If sent to WhyLabs, it will merge them automatically, creating statistical profiles on a daily basis.

Let's get our environment ready!¶

Uncomment the whylogs install if you don't have it with whylabs extension installed

In [1]:

# Note: you may need to restart the kernel to use updated packages.
%pip install pandas utils joblib scikit-learn Flask
%pip install whylogs

Note: you may need to restart the kernel to use updated packages.

In [2]:

import random
import numpy as np
import time
import requests
import pandas as pd
from joblib import dump
from sklearn.svm import SVC
import sklearn.datasets 
from sklearn.model_selection import train_test_split

Step 1: Load the Data¶

For this we will be using the IRIS dataset for our classification. It looks at the sepal and petal lengths and widths to be able to make a prediction of what species it is. This data set is readily available, but in this case we will grab it from sklearn's dataset library.

In [3]:

iris = sklearn.datasets.load_iris(as_frame=True)
data = pd.DataFrame(data=iris.data, columns=iris.feature_names)

data['target'] = [iris.target_names[i] for i in iris.target]
data

Out[3]:

	sepal length (cm)	sepal width (cm)	petal length (cm)	petal width (cm)	target
0	5.1	3.5	1.4	0.2	setosa
1	4.9	3.0	1.4	0.2	setosa
2	4.7	3.2	1.3	0.2	setosa
3	4.6	3.1	1.5	0.2	setosa
4	5.0	3.6	1.4	0.2	setosa
...	...	...	...	...	...
145	6.7	3.0	5.2	2.3	virginica
146	6.3	2.5	5.0	1.9	virginica
147	6.5	3.0	5.2	2.0	virginica
148	6.2	3.4	5.4	2.3	virginica
149	5.9	3.0	5.1	1.8	virginica

150 rows × 5 columns

In [4]:

       # Separating the independent variables from dependent variables
X = data.iloc[:, 0:4].values
y = data.iloc[:, -1].values
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)

Step 2: Train the Model¶

Next up, it's time to train the model. We will use a simple SVC, fit the modle, then dump it to "model.joblib".

In [5]:

# Train a classifier
print("Train started.")
model = SVC()
model.fit(x_train, y_train)
print("Train finished.")
# Save the model
dump(model, 'model.joblib')
print("Model saved as model.joblib")

Train started.
Train finished.
Model saved as model.joblib

Step 3: Build and Run a Docker Image¶

Within the directory this notebook is in you'll see all the code that makes up our flask app. There are a lot of files, but the main ones of interest are in the api folder. For this next step we will use docker to build an image based on all the requirements and settings that are put in that outer directory

In [6]:

!docker build --build-arg PYTHON_VERSION=3.9 -t whylogs-flask .

[+] Building 0.0s (0/1)                                                         
[+] Building 0.1s (2/3)                                                         
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 37B                                        0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 34B                                           0.0s
 => [internal] load metadata for docker.io/library/python:3.9              0.0s
[+] Building 0.3s (2/3)                                                         
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 37B                                        0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 34B                                           0.0s
 => [internal] load metadata for docker.io/library/python:3.9              0.2s
[+] Building 0.4s (2/3)                                                         
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 37B                                        0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 34B                                           0.0s
 => [internal] load metadata for docker.io/library/python:3.9              0.3s
[+] Building 0.6s (2/3)                                                         
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 37B                                        0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 34B                                           0.0s
 => [internal] load metadata for docker.io/library/python:3.9              0.5s
[+] Building 0.7s (2/3)                                                         
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 37B                                        0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 34B                                           0.0s
 => [internal] load metadata for docker.io/library/python:3.9              0.6s
[+] Building 0.9s (2/3)                                                         
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 37B                                        0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 34B                                           0.0s
 => [internal] load metadata for docker.io/library/python:3.9              0.8s
[+] Building 1.0s (2/3)                                                         
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 37B                                        0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 34B                                           0.0s
 => [internal] load metadata for docker.io/library/python:3.9              0.9s
[+] Building 1.2s (2/3)                                                         
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 37B                                        0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 34B                                           0.0s
 => [internal] load metadata for docker.io/library/python:3.9              1.1s
[+] Building 1.3s (2/3)                                                         
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 37B                                        0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 34B                                           0.0s
 => [internal] load metadata for docker.io/library/python:3.9              1.2s
[+] Building 1.4s (10/11)                                                       
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 37B                                        0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 34B                                           0.0s
 => [internal] load metadata for docker.io/library/python:3.9              1.2s
 => [1/6] FROM docker.io/library/python:3.9@sha256:51c996c8c65d826d20c613  0.0s
 => [internal] load build context                                          0.0s
 => => transferring context: 57.28kB                                       0.0s
 => CACHED [2/6] RUN mkdir /app                                            0.0s
 => CACHED [3/6] WORKDIR /app                                              0.0s
 => CACHED [4/6] COPY requirements.txt .                                   0.0s
 => CACHED [5/6] RUN pip install --no-cache-dir -r requirements.txt        0.0s
 => [6/6] COPY ./ /app                                                     0.0s
 => exporting to image                                                     0.0s
 => => exporting layers                                                    0.0s
[+] Building 1.4s (11/11) FINISHED                                              
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 37B                                        0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 34B                                           0.0s
 => [internal] load metadata for docker.io/library/python:3.9              1.2s
 => [1/6] FROM docker.io/library/python:3.9@sha256:51c996c8c65d826d20c613  0.0s
 => [internal] load build context                                          0.0s
 => => transferring context: 57.28kB                                       0.0s
 => CACHED [2/6] RUN mkdir /app                                            0.0s
 => CACHED [3/6] WORKDIR /app                                              0.0s
 => CACHED [4/6] COPY requirements.txt .                                   0.0s
 => CACHED [5/6] RUN pip install --no-cache-dir -r requirements.txt        0.0s
 => [6/6] COPY ./ /app                                                     0.0s
 => exporting to image                                                     0.0s
 => => exporting layers                                                    0.0s
 => => writing image sha256:33c622c8278ee63a8dd063de008fc364f5f33de4338ac  0.0s
 => => naming to docker.io/library/whylogs-flask                           0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them

Huzzah!! It built! We have a docker image, but nothing is running yet. Open a terminal and execute the following command:

docker run --rm -p 5000:5000 whylogs-flask

Step 4: Test Endpoint¶

Let's make sure it's actually up and running follow the directions below.

Go to http://0.0.0.0:5000/apidocs/
Open /predict endpoint green tab.
Click Try it out.
Click Execute green button.
Check the response and code, if 200, the API is working

If it's not working please check docker ps to see if something else is running on that port.

Step 5: Mess with Data to Showcase a Drift¶

Note that the logger is configured for to roll over every 5 min, we recommend running this for at least 15 min before really digging into the visualizations.

The following functions aim to modify the variables distribution in order to test whylabs.

In [7]:

def modify_random_column_values(data, value: float = np.random.uniform(low=0.0, high=10.0)) -> None:
    random_column = None
    data_mod = data.copy(deep=True)
    try:
        number_of_columns = len(data_mod.columns) - 2 # Index and label eliminated
        random_column = data_mod.columns[np.random.randint(number_of_columns) + 1]
        data_mod[random_column] = value
    except Exception as ex:
        raise f"Error adding fix value in random column: {str(random_column)}"
    return data_mod
        
        
def add_random_column_outliers(data, number_outliers: int = 10) -> None:
    random_column = None
    data_mod = data.copy(deep=True)
    try:
        number_of_columns = len(data_mod.columns) - 2  # Index and label eliminated
        number_of_rows = data_mod.shape[0]
        random_column = data_mod.columns[np.random.randint(number_of_columns) + 1]
        for i in range(number_outliers):
            random_row = np.random.randint(0, number_of_rows)
            data_mod.loc[random_row, random_column] = round(np.random.uniform(low=20.0, high=50.0), 2)
    except Exception as ex:
        raise f"Error adding outliers in random column: {random_column}"
    return data_mod

Once it's working, you can try to send continous requests to the endpoint:

In [8]:

labels = ["sepal_length_cm", "sepal_width_cm", "petal_length_cm", "petal_width_cm"]

In [9]:

# modify a variable distribution
data_mod = add_random_column_outliers(data, 30)
print("Dataset distribution modified!")

Dataset distribution modified!

In [10]:

url = "http://0.0.0.0:5000/api/v1"

In [11]:

healthy = requests.get(f"{url}/health")
if healthy.ok:
    for k in range(data_mod.shape[0]):
        # Build a payload with random values
        payload = dict(zip(labels, data_mod.iloc[:, 0:4].values[k]))
        print(payload)
        response = requests.post(f"{url}/predict", json=payload)
        if response.ok:
            print(response.json())
            time.sleep(random.randrange(2, 10))

{'sepal_length_cm': 5.1, 'sepal_width_cm': 3.5, 'petal_length_cm': 1.4, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 4.9, 'sepal_width_cm': 3.0, 'petal_length_cm': 1.4, 'petal_width_cm': 25.89}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 4.7, 'sepal_width_cm': 3.2, 'petal_length_cm': 1.3, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 4.6, 'sepal_width_cm': 3.1, 'petal_length_cm': 1.5, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.0, 'sepal_width_cm': 3.6, 'petal_length_cm': 1.4, 'petal_width_cm': 48.97}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 5.4, 'sepal_width_cm': 3.9, 'petal_length_cm': 1.7, 'petal_width_cm': 48.1}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 4.6, 'sepal_width_cm': 3.4, 'petal_length_cm': 1.4, 'petal_width_cm': 0.3}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.0, 'sepal_width_cm': 3.4, 'petal_length_cm': 1.5, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 4.4, 'sepal_width_cm': 2.9, 'petal_length_cm': 1.4, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 4.9, 'sepal_width_cm': 3.1, 'petal_length_cm': 1.5, 'petal_width_cm': 0.1}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.4, 'sepal_width_cm': 3.7, 'petal_length_cm': 1.5, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 4.8, 'sepal_width_cm': 3.4, 'petal_length_cm': 1.6, 'petal_width_cm': 26.61}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 4.8, 'sepal_width_cm': 3.0, 'petal_length_cm': 1.4, 'petal_width_cm': 0.1}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 4.3, 'sepal_width_cm': 3.0, 'petal_length_cm': 1.1, 'petal_width_cm': 43.34}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 5.8, 'sepal_width_cm': 4.0, 'petal_length_cm': 1.2, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.7, 'sepal_width_cm': 4.4, 'petal_length_cm': 1.5, 'petal_width_cm': 0.4}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.4, 'sepal_width_cm': 3.9, 'petal_length_cm': 1.3, 'petal_width_cm': 0.4}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.1, 'sepal_width_cm': 3.5, 'petal_length_cm': 1.4, 'petal_width_cm': 0.3}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.7, 'sepal_width_cm': 3.8, 'petal_length_cm': 1.7, 'petal_width_cm': 0.3}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.1, 'sepal_width_cm': 3.8, 'petal_length_cm': 1.5, 'petal_width_cm': 0.3}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.4, 'sepal_width_cm': 3.4, 'petal_length_cm': 1.7, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.1, 'sepal_width_cm': 3.7, 'petal_length_cm': 1.5, 'petal_width_cm': 0.4}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 4.6, 'sepal_width_cm': 3.6, 'petal_length_cm': 1.0, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.1, 'sepal_width_cm': 3.3, 'petal_length_cm': 1.7, 'petal_width_cm': 0.5}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 4.8, 'sepal_width_cm': 3.4, 'petal_length_cm': 1.9, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.0, 'sepal_width_cm': 3.0, 'petal_length_cm': 1.6, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.0, 'sepal_width_cm': 3.4, 'petal_length_cm': 1.6, 'petal_width_cm': 0.4}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.2, 'sepal_width_cm': 3.5, 'petal_length_cm': 1.5, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.2, 'sepal_width_cm': 3.4, 'petal_length_cm': 1.4, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 4.7, 'sepal_width_cm': 3.2, 'petal_length_cm': 1.6, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 4.8, 'sepal_width_cm': 3.1, 'petal_length_cm': 1.6, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.4, 'sepal_width_cm': 3.4, 'petal_length_cm': 1.5, 'petal_width_cm': 0.4}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.2, 'sepal_width_cm': 4.1, 'petal_length_cm': 1.5, 'petal_width_cm': 0.1}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.5, 'sepal_width_cm': 4.2, 'petal_length_cm': 1.4, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 4.9, 'sepal_width_cm': 3.1, 'petal_length_cm': 1.5, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.0, 'sepal_width_cm': 3.2, 'petal_length_cm': 1.2, 'petal_width_cm': 27.27}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 5.5, 'sepal_width_cm': 3.5, 'petal_length_cm': 1.3, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 4.9, 'sepal_width_cm': 3.6, 'petal_length_cm': 1.4, 'petal_width_cm': 0.1}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 4.4, 'sepal_width_cm': 3.0, 'petal_length_cm': 1.3, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.1, 'sepal_width_cm': 3.4, 'petal_length_cm': 1.5, 'petal_width_cm': 28.7}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 5.0, 'sepal_width_cm': 3.5, 'petal_length_cm': 1.3, 'petal_width_cm': 0.3}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 4.5, 'sepal_width_cm': 2.3, 'petal_length_cm': 1.3, 'petal_width_cm': 0.3}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 4.4, 'sepal_width_cm': 3.2, 'petal_length_cm': 1.3, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.0, 'sepal_width_cm': 3.5, 'petal_length_cm': 1.6, 'petal_width_cm': 45.86}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 5.1, 'sepal_width_cm': 3.8, 'petal_length_cm': 1.9, 'petal_width_cm': 41.23}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 4.8, 'sepal_width_cm': 3.0, 'petal_length_cm': 1.4, 'petal_width_cm': 0.3}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.1, 'sepal_width_cm': 3.8, 'petal_length_cm': 1.6, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 4.6, 'sepal_width_cm': 3.2, 'petal_length_cm': 1.4, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.3, 'sepal_width_cm': 3.7, 'petal_length_cm': 1.5, 'petal_width_cm': 0.2}
{'data': {'class': 'setosa'}, 'message': 'Success'}
{'sepal_length_cm': 5.0, 'sepal_width_cm': 3.3, 'petal_length_cm': 1.4, 'petal_width_cm': 25.96}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 7.0, 'sepal_width_cm': 3.2, 'petal_length_cm': 4.7, 'petal_width_cm': 1.4}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 6.4, 'sepal_width_cm': 3.2, 'petal_length_cm': 4.5, 'petal_width_cm': 1.5}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 6.9, 'sepal_width_cm': 3.1, 'petal_length_cm': 4.9, 'petal_width_cm': 1.5}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 5.5, 'sepal_width_cm': 2.3, 'petal_length_cm': 4.0, 'petal_width_cm': 31.79}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.5, 'sepal_width_cm': 2.8, 'petal_length_cm': 4.6, 'petal_width_cm': 1.5}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 5.7, 'sepal_width_cm': 2.8, 'petal_length_cm': 4.5, 'petal_width_cm': 32.73}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.3, 'sepal_width_cm': 3.3, 'petal_length_cm': 4.7, 'petal_width_cm': 1.6}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 4.9, 'sepal_width_cm': 2.4, 'petal_length_cm': 3.3, 'petal_width_cm': 1.0}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 6.6, 'sepal_width_cm': 2.9, 'petal_length_cm': 4.6, 'petal_width_cm': 1.3}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 5.2, 'sepal_width_cm': 2.7, 'petal_length_cm': 3.9, 'petal_width_cm': 30.06}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 5.0, 'sepal_width_cm': 2.0, 'petal_length_cm': 3.5, 'petal_width_cm': 1.0}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 5.9, 'sepal_width_cm': 3.0, 'petal_length_cm': 4.2, 'petal_width_cm': 1.5}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 6.0, 'sepal_width_cm': 2.2, 'petal_length_cm': 4.0, 'petal_width_cm': 1.0}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 6.1, 'sepal_width_cm': 2.9, 'petal_length_cm': 4.7, 'petal_width_cm': 1.4}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 5.6, 'sepal_width_cm': 2.9, 'petal_length_cm': 3.6, 'petal_width_cm': 47.54}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.7, 'sepal_width_cm': 3.1, 'petal_length_cm': 4.4, 'petal_width_cm': 1.4}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 5.6, 'sepal_width_cm': 3.0, 'petal_length_cm': 4.5, 'petal_width_cm': 44.65}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 5.8, 'sepal_width_cm': 2.7, 'petal_length_cm': 4.1, 'petal_width_cm': 1.0}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 6.2, 'sepal_width_cm': 2.2, 'petal_length_cm': 4.5, 'petal_width_cm': 1.5}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 5.6, 'sepal_width_cm': 2.5, 'petal_length_cm': 3.9, 'petal_width_cm': 1.1}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 5.9, 'sepal_width_cm': 3.2, 'petal_length_cm': 4.8, 'petal_width_cm': 1.8}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.1, 'sepal_width_cm': 2.8, 'petal_length_cm': 4.0, 'petal_width_cm': 1.3}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 6.3, 'sepal_width_cm': 2.5, 'petal_length_cm': 4.9, 'petal_width_cm': 1.5}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 6.1, 'sepal_width_cm': 2.8, 'petal_length_cm': 4.7, 'petal_width_cm': 1.2}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 6.4, 'sepal_width_cm': 2.9, 'petal_length_cm': 4.3, 'petal_width_cm': 1.3}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 6.6, 'sepal_width_cm': 3.0, 'petal_length_cm': 4.4, 'petal_width_cm': 1.4}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 6.8, 'sepal_width_cm': 2.8, 'petal_length_cm': 4.8, 'petal_width_cm': 1.4}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 6.7, 'sepal_width_cm': 3.0, 'petal_length_cm': 5.0, 'petal_width_cm': 32.12}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.0, 'sepal_width_cm': 2.9, 'petal_length_cm': 4.5, 'petal_width_cm': 1.5}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 5.7, 'sepal_width_cm': 2.6, 'petal_length_cm': 3.5, 'petal_width_cm': 1.0}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 5.5, 'sepal_width_cm': 2.4, 'petal_length_cm': 3.8, 'petal_width_cm': 1.1}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 5.5, 'sepal_width_cm': 2.4, 'petal_length_cm': 3.7, 'petal_width_cm': 1.0}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 5.8, 'sepal_width_cm': 2.7, 'petal_length_cm': 3.9, 'petal_width_cm': 43.94}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.0, 'sepal_width_cm': 2.7, 'petal_length_cm': 5.1, 'petal_width_cm': 1.6}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 5.4, 'sepal_width_cm': 3.0, 'petal_length_cm': 4.5, 'petal_width_cm': 1.5}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 6.0, 'sepal_width_cm': 3.4, 'petal_length_cm': 4.5, 'petal_width_cm': 1.6}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 6.7, 'sepal_width_cm': 3.1, 'petal_length_cm': 4.7, 'petal_width_cm': 1.5}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 6.3, 'sepal_width_cm': 2.3, 'petal_length_cm': 4.4, 'petal_width_cm': 49.06}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 5.6, 'sepal_width_cm': 3.0, 'petal_length_cm': 4.1, 'petal_width_cm': 1.3}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 5.5, 'sepal_width_cm': 2.5, 'petal_length_cm': 4.0, 'petal_width_cm': 1.3}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 5.5, 'sepal_width_cm': 2.6, 'petal_length_cm': 4.4, 'petal_width_cm': 1.2}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 6.1, 'sepal_width_cm': 3.0, 'petal_length_cm': 4.6, 'petal_width_cm': 1.4}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 5.8, 'sepal_width_cm': 2.6, 'petal_length_cm': 4.0, 'petal_width_cm': 32.23}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 5.0, 'sepal_width_cm': 2.3, 'petal_length_cm': 3.3, 'petal_width_cm': 1.0}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 5.6, 'sepal_width_cm': 2.7, 'petal_length_cm': 4.2, 'petal_width_cm': 1.3}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 5.7, 'sepal_width_cm': 3.0, 'petal_length_cm': 4.2, 'petal_width_cm': 1.2}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 5.7, 'sepal_width_cm': 2.9, 'petal_length_cm': 4.2, 'petal_width_cm': 48.92}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.2, 'sepal_width_cm': 2.9, 'petal_length_cm': 4.3, 'petal_width_cm': 1.3}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 5.1, 'sepal_width_cm': 2.5, 'petal_length_cm': 3.0, 'petal_width_cm': 1.1}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 5.7, 'sepal_width_cm': 2.8, 'petal_length_cm': 4.1, 'petal_width_cm': 1.3}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 6.3, 'sepal_width_cm': 3.3, 'petal_length_cm': 6.0, 'petal_width_cm': 2.5}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 5.8, 'sepal_width_cm': 2.7, 'petal_length_cm': 5.1, 'petal_width_cm': 1.9}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 7.1, 'sepal_width_cm': 3.0, 'petal_length_cm': 5.9, 'petal_width_cm': 33.89}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.3, 'sepal_width_cm': 2.9, 'petal_length_cm': 5.6, 'petal_width_cm': 1.8}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.5, 'sepal_width_cm': 3.0, 'petal_length_cm': 5.8, 'petal_width_cm': 33.77}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 7.6, 'sepal_width_cm': 3.0, 'petal_length_cm': 6.6, 'petal_width_cm': 2.1}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 4.9, 'sepal_width_cm': 2.5, 'petal_length_cm': 4.5, 'petal_width_cm': 1.7}
{'data': {'class': 'versicolor'}, 'message': 'Success'}
{'sepal_length_cm': 7.3, 'sepal_width_cm': 2.9, 'petal_length_cm': 6.3, 'petal_width_cm': 1.8}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.7, 'sepal_width_cm': 2.5, 'petal_length_cm': 5.8, 'petal_width_cm': 25.81}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 7.2, 'sepal_width_cm': 3.6, 'petal_length_cm': 6.1, 'petal_width_cm': 2.5}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.5, 'sepal_width_cm': 3.2, 'petal_length_cm': 5.1, 'petal_width_cm': 2.0}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.4, 'sepal_width_cm': 2.7, 'petal_length_cm': 5.3, 'petal_width_cm': 1.9}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.8, 'sepal_width_cm': 3.0, 'petal_length_cm': 5.5, 'petal_width_cm': 2.1}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 5.7, 'sepal_width_cm': 2.5, 'petal_length_cm': 5.0, 'petal_width_cm': 34.12}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 5.8, 'sepal_width_cm': 2.8, 'petal_length_cm': 5.1, 'petal_width_cm': 2.4}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.4, 'sepal_width_cm': 3.2, 'petal_length_cm': 5.3, 'petal_width_cm': 2.3}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.5, 'sepal_width_cm': 3.0, 'petal_length_cm': 5.5, 'petal_width_cm': 37.76}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 7.7, 'sepal_width_cm': 3.8, 'petal_length_cm': 6.7, 'petal_width_cm': 2.2}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 7.7, 'sepal_width_cm': 2.6, 'petal_length_cm': 6.9, 'petal_width_cm': 2.3}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.0, 'sepal_width_cm': 2.2, 'petal_length_cm': 5.0, 'petal_width_cm': 1.5}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.9, 'sepal_width_cm': 3.2, 'petal_length_cm': 5.7, 'petal_width_cm': 2.3}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 5.6, 'sepal_width_cm': 2.8, 'petal_length_cm': 4.9, 'petal_width_cm': 2.0}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 7.7, 'sepal_width_cm': 2.8, 'petal_length_cm': 6.7, 'petal_width_cm': 44.82}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.3, 'sepal_width_cm': 2.7, 'petal_length_cm': 4.9, 'petal_width_cm': 1.8}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.7, 'sepal_width_cm': 3.3, 'petal_length_cm': 5.7, 'petal_width_cm': 2.1}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 7.2, 'sepal_width_cm': 3.2, 'petal_length_cm': 6.0, 'petal_width_cm': 1.8}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.2, 'sepal_width_cm': 2.8, 'petal_length_cm': 4.8, 'petal_width_cm': 1.8}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.1, 'sepal_width_cm': 3.0, 'petal_length_cm': 4.9, 'petal_width_cm': 1.8}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.4, 'sepal_width_cm': 2.8, 'petal_length_cm': 5.6, 'petal_width_cm': 2.1}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 7.2, 'sepal_width_cm': 3.0, 'petal_length_cm': 5.8, 'petal_width_cm': 1.6}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 7.4, 'sepal_width_cm': 2.8, 'petal_length_cm': 6.1, 'petal_width_cm': 40.22}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 7.9, 'sepal_width_cm': 3.8, 'petal_length_cm': 6.4, 'petal_width_cm': 2.0}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.4, 'sepal_width_cm': 2.8, 'petal_length_cm': 5.6, 'petal_width_cm': 2.2}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.3, 'sepal_width_cm': 2.8, 'petal_length_cm': 5.1, 'petal_width_cm': 1.5}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.1, 'sepal_width_cm': 2.6, 'petal_length_cm': 5.6, 'petal_width_cm': 1.4}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 7.7, 'sepal_width_cm': 3.0, 'petal_length_cm': 6.1, 'petal_width_cm': 2.3}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.3, 'sepal_width_cm': 3.4, 'petal_length_cm': 5.6, 'petal_width_cm': 2.4}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.4, 'sepal_width_cm': 3.1, 'petal_length_cm': 5.5, 'petal_width_cm': 1.8}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.0, 'sepal_width_cm': 3.0, 'petal_length_cm': 4.8, 'petal_width_cm': 1.8}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.9, 'sepal_width_cm': 3.1, 'petal_length_cm': 5.4, 'petal_width_cm': 2.1}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.7, 'sepal_width_cm': 3.1, 'petal_length_cm': 5.6, 'petal_width_cm': 2.4}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.9, 'sepal_width_cm': 3.1, 'petal_length_cm': 5.1, 'petal_width_cm': 49.55}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 5.8, 'sepal_width_cm': 2.7, 'petal_length_cm': 5.1, 'petal_width_cm': 1.9}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.8, 'sepal_width_cm': 3.2, 'petal_length_cm': 5.9, 'petal_width_cm': 2.3}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.7, 'sepal_width_cm': 3.3, 'petal_length_cm': 5.7, 'petal_width_cm': 2.5}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.7, 'sepal_width_cm': 3.0, 'petal_length_cm': 5.2, 'petal_width_cm': 2.3}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.3, 'sepal_width_cm': 2.5, 'petal_length_cm': 5.0, 'petal_width_cm': 1.9}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.5, 'sepal_width_cm': 3.0, 'petal_length_cm': 5.2, 'petal_width_cm': 2.0}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 6.2, 'sepal_width_cm': 3.4, 'petal_length_cm': 5.4, 'petal_width_cm': 2.3}
{'data': {'class': 'virginica'}, 'message': 'Success'}
{'sepal_length_cm': 5.9, 'sepal_width_cm': 3.0, 'petal_length_cm': 5.1, 'petal_width_cm': 1.8}
{'data': {'class': 'virginica'}, 'message': 'Success'}

Viewing the Data¶

This app defaults to a local file writer. You can look at the logs within the docker container by running docker exec -it <container-id> bash . They will all be in ls /logs

While there you can visualize the results by following the instructions of the "Notebook Profile Visualizer" . Some more information can be found in these notebooks "Merging Profiles" and "Streaming_Data_with_Log_Rotation".

Using WhyLabs¶

As we talked about in the overview though we will also be able to send this over to your WhyLabs observation dashboard. Here it will automatically receive the data and merge them into useful daily analysis. Follow the directions in "WhyLabs Writer" to change the .env file in this directory. Then rerun the above. This time it will populate your WhyLabs dataset to be able to see the drifts on the WhyLabs portal.

Included in this notebook's directory is a .env file with the needed variables set to None.

WhyLabs Profile

In [ ]:

🚩 Create a free WhyLabs account to get more value out of whylogs!