Notebook

In [1]:

from orion.data import load_signal, load_anomalies

1. Data¶

In [2]:

signal_name = 'S-1'

data = load_signal(signal_name)

anomalies = load_anomalies(signal_name)

data.head()

Out[2]:

	timestamp	value
0	1222819200	-0.366359
1	1222840800	-0.394108
2	1222862400	0.403625
3	1222884000	-0.362759
4	1222905600	-0.370746

2. Pipeline¶

In [3]:

from mlblocks import MLPipeline

pipeline_name = 'timesfm'

pipeline = MLPipeline(pipeline_name)

TimesFM v1.2.0. See https://github.com/google-research/timesfm/blob/master/README.md for updated APIs.
Loaded Jax TimesFM.
Loaded PyTorch TimesFM.

/home/linhnk/Orion/.conda/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
Fetching 3 files: 100%|██████████| 3/3 [00:00<00:00, 11759.73it/s]

step by step execution¶

MLPipelines are compose of a squence of primitives, these primitives apply tranformation and calculation operations to the data and updates the variables within the pipeline. To view the primitives used by the pipeline, we access its primtivies attribute.

The UniTS contains 6 primitives. we will observe how the context (which are the variables held within the pipeline) are updated after the execution of each primitive.

In [4]:

pipeline.primitives

Out[4]:

['mlstars.custom.timeseries_preprocessing.time_segments_aggregate',
 'sklearn.impute.SimpleImputer',
 'sklearn.preprocessing.StandardScaler',
 'mlstars.custom.timeseries_preprocessing.rolling_window_sequences',
 'orion.primitives.timesfm.TimesFM',
 'orion.primitives.timeseries_errors.regression_errors',
 'orion.primitives.timeseries_anomalies.find_anomalies']

time segments aggregate¶

this primitive creates an equi-spaced time series by aggregating values over fixed specified interval.

input: X which is an n-dimensional sequence of values.
output:
- X sequence of aggregated values, one column for each aggregation method.
- index sequence of index values (first index of each aggregated segment).

In [5]:

context = pipeline.fit(data, output_=0)
context.keys()

Out[5]:

dict_keys(['X', 'index'])

In [6]:

for i, x in list(zip(context['index'], context['X']))[:5]:
    print("entry at {} has value {}".format(i, x))

entry at 1222819200 has value [-0.36635895]
entry at 1222840800 has value [-0.39410778]
entry at 1222862400 has value [0.4036246]
entry at 1222884000 has value [-0.36275906]
entry at 1222905600 has value [-0.37074649]

SimpleImputer¶

this primitive is an imputation transformer for filling missing values.

input: X which is an n-dimensional sequence of values.
output: X which is a transformed version of X.

In [7]:

step = 1

context = pipeline.fit(**context, output_=step, start_=step)
context.keys()

Out[7]:

dict_keys(['index', 'X'])

StandardScaler¶

this primitive transforms features by scaling each feature to a given range.

input: X the data used to compute the per-feature mean and standard deviation used for later scaling along the features axis.
output: X which is a transformed version of X.

In [8]:

step = 2

context = pipeline.fit(**context, output_=step, start_=step)
context.keys()

Out[8]:

dict_keys(['index', 'X'])

In [9]:

# after scaling the data between [-1, 1]

for i, x in list(zip(context['index'], context['X']))[:5]:
    print("entry at {} has value {}".format(i, x))

entry at 1222819200 has value [-0.4818325]
entry at 1222840800 has value [-0.54231192]
entry at 1222862400 has value [1.19636972]
entry at 1222884000 has value [-0.47398644]
entry at 1222905600 has value [-0.49139529]

rolling window sequence¶

this primitive generates many sub-sequences of the original sequence. it uses a rolling window approach to create the sub-sequences out of time series data.

input:
- X n-dimensional sequence to iterate over.
- index array containing the index values of X.
output:
- X input sequences.
- y target sequences.
- index first index value of each input sequence.
- target_index first index value of each target sequence.

In [10]:

step = 3

context = pipeline.fit(**context, output_=step, start_=step)
context.keys()

Out[10]:

dict_keys(['index', 'X', 'y', 'target_index'])

In [11]:

# after slicing X into multiple sub-sequences
# we obtain a 3 dimensional matrix X where
# the shape indicates (# slices, window size, 1)
# and similarly y is (# slices, target size)

print("X shape = {}\ny shape = {}\nindex shape = {}\ntarget index shape = {}".format(
    context['X'].shape, context['y'].shape, context['index'].shape, context['target_index'].shape))

X shape = (9893, 256, 1)
y shape = (9893, 1)
index shape = (9893,)
target index shape = (9893,)

TimesFM¶

this is a forecasting using TimesFM. you can read more about it in the related paper.

input:
- X n-dimensional array containing the input sequences for the model.
output:
- y_hat predicted values

In [12]:

step = 4

context = pipeline.fit(**context, output_=step, start_=step)
context.keys()

Out[12]:

dict_keys(['index', 'target_index', 'X', 'y', 'y_hat'])

In [13]:

context['y_hat'].shape

Out[13]:

(9893,)

regression errors¶

this primitive computes an array of errors comparing predictions and expected output.

input:
- y ground truth.
- y_hat forecasted values.
output: errors array of errors.

In [14]:

step = 5

context = pipeline.fit(**context, output_=step, start_=step)
context.keys()

Out[14]:

dict_keys(['index', 'target_index', 'y_hat', 'X', 'y', 'errors'])

find anomalies¶

this primitive finds anomalies from sequence of errors

input:
- errors array of errors
- target_index indices
output: anomalies.

In [15]:

step = 6

context = pipeline.fit(**context, output_=step, start_=step)
context.keys()

/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)
/home/linhnk/Orion/.conda/lib/python3.11/site-packages/scipy/optimize/_optimize.py:836: RuntimeWarning: invalid value encountered in subtract
  np.max(np.abs(fsim[0] - fsim[1:])) <= fatol):
/home/linhnk/Orion/orion/primitives/timeseries_anomalies.py:196: FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated and will change in a future version. Call result.infer_objects(copy=False) instead. To opt-in to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
  shift = above.shift(1).fillna(False)

Out[15]:

dict_keys(['index', 'target_index', 'y_hat', 'errors', 'X', 'y', 'anomalies'])

In [16]:

context['anomalies']

Out[16]:

array([[1.3985352e+09, 1.4064624e+09, 4.5702480e-01]])

3. Evaluate performance¶

In this next step we will load some already known anomalous intervals and evaluate how good our anomaly detection was by comparing those with our detected intervals.

For this, we will first load the known anomalies for the signal that we are using:

In [19]:

from orion.data import load_anomalies

ground_truth = load_anomalies('S-1')
ground_truth

Out[19]:

	start	end
0	1398168000	1407823200

In [20]:

anomalies = []
for ano in context['anomalies']:
    anomalies.append((ano[0], ano[1]))
anomalies

Out[20]:

[(1398535200.0, 1406462400.0)]

In [21]:

from orion.evaluation import contextual_confusion_matrix, contextual_f1_score

start, end = context['index'][0], context['index'][-1]

contextual_confusion_matrix(ground_truth, anomalies, start = start, end = end, weighted=False)

Out[21]:

(None, 0, 0, 1)

In [22]:

contextual_f1_score(ground_truth, anomalies, start = start, end = end, weighted=False)

Out[22]:

1.0

In [ ]: