Converting Profiles from whylogs v0 to v1¶

If, for some reason, you have profiles generated from whylogs v0 (Python or Java) and wish to work with them in whylogs v1, we provide converters to help you do so.

Once you convert the profiles to v1, you can use them just as you would any other v1 whylogs profile.

This short example is divided into two parts:

Download a sample v0 profile and write it to disk
Read the v0 profile and convert it to a v1 Profile View

Let's get to it!

Installing whylogs and importing modules¶

In [ ]:

# Note: you may need to restart the kernel to use updated packages.
%pip install whylogs

Downloading v0 profile¶

First, we need a sample v0 profile to demonstrate how to convert it.

To do so, we'll download a v0 profile from S3 and write it to disk.

In [1]:

#write a file to disk from an url
from urllib.request import urlopen
url = "https://whylabs-public.s3.us-west-2.amazonaws.com/whylogs_examples/dataset_profile_v0.bin"
profile_name = "dataset_profile_v0.bin"

# Download from URL
with urlopen(url) as file:
    content = file.read()

# Save to file
with open(profile_name, 'wb') as download:
    download.write(content)

Convert serialized v0 profile to v1 profile view¶

The converter will enable you to read the v0 profile and convert it to a v1 profile view.

Considering it's a Profile View, you'll be able to use it for tasks such as visualization, analysis and merging. However, you won't be able to use it to continue logging new data.

To do so, we'll use the read_v0_to_view utility from whylogs.migration.converters.

In [2]:

from whylogs.migration.converters import (
    read_v0_to_view
)
from whylogs.core import DatasetProfileView

import whylogs as why

print(f"For this example to work you need to have a recent version of whylogs (tested with 1.1.22), you are currently running: whylogs=={why.__version__}")

profile_file_path_v0 = "dataset_profile_v0.bin"

print(f"Reading v0 file from disk: {profile_file_path_v0}")
view: DatasetProfileView = read_v0_to_view(profile_file_path_v0)
print(f"Converted: {profile_file_path_v0} to a v1 DatasetProfileView")
view.to_pandas()

For this example to work you need to have a recent version of whylogs (tested with 1.1.22), you are currently running: whylogs==1.1.19
Reading v0 file from disk: dataset_profile_v0.bin
Converted: dataset_profile_v0.bin to a v1 DatasetProfileView

Out[2]:

	cardinality/est	cardinality/lower_1	cardinality/upper_1	counts/inf	counts/n	counts/nan	counts/null	distribution/max	distribution/mean	distribution/median	...	distribution/stddev	frequent_items/frequent_strings	ints/max	ints/min	type	types/boolean	types/fractional	types/integral	types/object	types/string
column
animal	3.0	3.0	3.00015	0	4	0	0	NaN	0.0	NaN	...	0.000000	[FrequentItem(value='cat', est=2, upper=2, low...	0	0	SummaryType.COLUMN	0	0	0	0	4
legs	3.0	3.0	3.00015	0	4	0	0	4.0	2.5	4.0	...	1.914854	[FrequentItem(value='4', est=2, upper=2, lower...	0	0	SummaryType.COLUMN	0	0	4	0	0
weight	3.0	3.0	3.00015	0	4	0	0	4.3	3.4	4.1	...	1.389244	[FrequentItem(value='1.8', est=1, upper=1, low...	0	0	SummaryType.COLUMN	0	3	0	0	0

3 rows × 30 columns

And there you have it!

You can now use the profile for tasks such as:

Head to our examples page to see more!