🚩 Create a free WhyLabs account to get more value out of whylogs!
Did you know you can store, visualize, and monitor whylogs profiles with the WhyLabs Observability Platform? Sign up for a free WhyLabs account to leverage the power of whylogs and WhyLabs together!
Custom Performance Metrics¶
If the default performance metrics such as accuracy, are not sufficient for your project's needs, you can add your own metrics and designate them as Custom Performance Metrics. This allows you to use features such as Segment Analysis and Performance Comparison with metrics that are more relevant to your use case.
Let's start by installing and initializing whylogs.
%pip install whylogs
import whylogs as why
why.init()
Now we can log the custom metric's value and tag it as a custom metric using the tag_custom_performance_column() method. This will make the metric show up in the Dashboards and Performance section. The "mean" will be shown there by default, and values will be treated like single point observations, so if you log two different values in the same time window for the same custom performance metrics, you will get a numeric distribution of those datapoints, not overwriting the last value. We suggest only logging one value per time window when using custom performance metrics in this way, or to be aware of the way multiple uploads in the same time.
from whylogs.api.writer.whylabs import WhyLabsWriter
perf_metric_name = "custom_f1"
results = why.log({perf_metric_name: 0.875})
writer = WhyLabsWriter()
writer.write(results.view())
writer.tag_custom_performance_column(column=perf_metric_name)
