Notebook

Use Case: Unmatched Instances Input¶

Install Dependencies¶

In [1]:

!pip install -r requirements.txt

Requirement already satisfied: panoptica in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from -r requirements.txt (line 1)) (1.0.0.post2.dev0+2f7d01f)
Requirement already satisfied: auxiliary in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from -r requirements.txt (line 2)) (0.0.42)
Requirement already satisfied: rich in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from -r requirements.txt (line 3)) (13.6.0)
Requirement already satisfied: numpy in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from -r requirements.txt (line 4)) (1.25.2)
Requirement already satisfied: connected-components-3d<4.0.0,>=3.12.3 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from panoptica->-r requirements.txt (line 1)) (3.12.3)
Requirement already satisfied: ruamel.yaml<0.19.0,>=0.18.6 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from panoptica->-r requirements.txt (line 1)) (0.18.6)
Requirement already satisfied: scikit-image<0.23.0,>=0.22.0 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from panoptica->-r requirements.txt (line 1)) (0.22.0)
Requirement already satisfied: scipy<2.0.0,>=1.7.0 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from panoptica->-r requirements.txt (line 1)) (1.11.2)
Requirement already satisfied: nibabel>=3.0 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from auxiliary->-r requirements.txt (line 2)) (5.1.0)
Requirement already satisfied: path>=16.10.0 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from auxiliary->-r requirements.txt (line 2)) (17.0.0)
Requirement already satisfied: pathlib>=1.0 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from auxiliary->-r requirements.txt (line 2)) (1.0.1)
Requirement already satisfied: pillow>=10.0.0 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from auxiliary->-r requirements.txt (line 2)) (10.0.0)
Requirement already satisfied: tifffile>=2023.8.25 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from auxiliary->-r requirements.txt (line 2)) (2023.8.30)
Requirement already satisfied: markdown-it-py>=2.2.0 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from rich->-r requirements.txt (line 3)) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from rich->-r requirements.txt (line 3)) (2.17.2)
Requirement already satisfied: mdurl~=0.1 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from markdown-it-py>=2.2.0->rich->-r requirements.txt (line 3)) (0.1.2)
Requirement already satisfied: packaging>=17 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from nibabel>=3.0->auxiliary->-r requirements.txt (line 2)) (23.1)
Requirement already satisfied: ruamel.yaml.clib>=0.2.7 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from ruamel.yaml<0.19.0,>=0.18.6->panoptica->-r requirements.txt (line 1)) (0.2.8)
Requirement already satisfied: networkx>=2.8 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from scikit-image<0.23.0,>=0.22.0->panoptica->-r requirements.txt (line 1)) (3.1)
Requirement already satisfied: imageio>=2.27 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from scikit-image<0.23.0,>=0.22.0->panoptica->-r requirements.txt (line 1)) (2.31.3)
Requirement already satisfied: lazy_loader>=0.3 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from scikit-image<0.23.0,>=0.22.0->panoptica->-r requirements.txt (line 1)) (0.3)

Setup Imports¶

In [2]:

import numpy as np
from auxiliary.nifti.io import read_nifti
from rich import print as pprint
from panoptica import NaiveThresholdMatching, Panoptica_Evaluator, InputType
from panoptica.utils.segmentation_class import LabelGroup, SegmentationClassGroups

Load Example Data¶

To demonstrate we use a reference and predicition of spine a segmentation with unmatched instances.

unmatched_instance_figure

In [3]:

ref_masks = read_nifti("./spine_seg/unmatched_instance/ref.nii.gz")
pred_masks = read_nifti("./spine_seg/unmatched_instance/pred.nii.gz")

# labels are unmatched instances
pred_masks[pred_masks == 27] = 26  # For later
np.unique(ref_masks), np.unique(pred_masks)

Out[3]:

(array([  0,   2,   3,   4,   5,   6,   7,   8,  26, 102, 103, 104, 105,
        106, 107, 108, 202, 203, 204, 205, 206, 207, 208], dtype=uint8),
 array([  0,   3,   4,   5,   6,   7,   8,   9,  26, 103, 104, 105, 106,
        107, 108, 109, 203, 204, 205, 206, 207, 208, 209], dtype=uint8))

Define Class Groups¶

In [4]:

# Define (optionally) semantic groups
# This means that only instance within one group can be matched to each other
segmentation_class_groups = SegmentationClassGroups(
    {
        "vertebra": LabelGroup(list(range(1, 11))),
        "ivd": LabelGroup(list(range(101, 111))),
        "sacrum": ([26], True),
        "endplate": LabelGroup(list(range(201, 211))),
    }
)
# In this case, the label 26 can only be matched with label 26 (thats why have to ensure above that 26 exists in both masks, otherwise they wouldn't be matched)

Let's do it ourselves!¶

Panoptica allows you to call everything yourself if you really want to

In [5]:

# Input are unmatched instances, so lets match em!
from panoptica import Metric

# This will match based on IoU metric, will only match if instance have a IoU of 0.5 or higher and will not allow multiple predictions to be matched to the same reference
matcher = NaiveThresholdMatching(
    matching_metric=Metric.IOU, matching_threshold=0.5, allow_many_to_one=False
)

# Now we have to do our processing object ourselves
from panoptica import UnmatchedInstancePair

unmatched_instance_input = UnmatchedInstancePair(pred_masks, ref_masks)

matched_instance_output = matcher.match_instances(unmatched_instance_input)
print("prediction_arr=", np.unique(matched_instance_output.prediction_arr))
print("reference_arr=", np.unique(matched_instance_output.reference_arr))

# Based of this, we see that some references are not sucessfully hit (203, 205, 208)
# We can also see that we indeed have the same number of prediction instances that got no match, they will be appended afterwards (209, 210, 211)

prediction_arr= [  0   2   3   4   5   6   7   8  26 102 103 104 105 106 107 108 202 204
 206 207 209 210 211]
reference_arr= [  0   2   3   4   5   6   7   8  26 102 103 104 105 106 107 108 202 203
 204 205 206 207 208]

Let's match 'em all!¶

In [6]:

# This will match based on IoU metric, will only match if instance have a IoU of 0.0 or higher and will not allow multiple predictions to be matched to the same reference
matcher = NaiveThresholdMatching(
    matching_metric=Metric.IOU, matching_threshold=0.0, allow_many_to_one=False
)

matched_instance_output = matcher.match_instances(unmatched_instance_input)
print("prediction_arr=", np.unique(matched_instance_output.prediction_arr))
print("reference_arr=", np.unique(matched_instance_output.reference_arr))

# With a threshold of 0.0, we ensure that we match as much as possible.
# We see, that contrary to before, instances 203, 205, and 208 are now matched

prediction_arr= [  0   2   3   4   5   6   7   8  26 102 103 104 105 106 107 108 202 203
 204 205 206 207 208]
reference_arr= [  0   2   3   4   5   6   7   8  26 102 103 104 105 106 107 108 202 203
 204 205 206 207 208]

Do it yourself¶

Now it is up to you to explore the different matching algorithms and the best setup for your project

Just remember, this setup can have drastic differences in the resulting metrics as well as interpretation of those results. For example, if you always match everything, of course your F1-Score will be 1.0. This becomes meaningless then. Also the choice of metric does matter!