Semantic Segmentation Input¶
Install Dependencies¶
In [1]:
!pip install -r requirements.txt
Requirement already satisfied: panoptica in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from -r requirements.txt (line 1)) (1.0.0.post2.dev0+2f7d01f) Requirement already satisfied: auxiliary in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from -r requirements.txt (line 2)) (0.0.42) Requirement already satisfied: rich in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from -r requirements.txt (line 3)) (13.6.0) Requirement already satisfied: numpy in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from -r requirements.txt (line 4)) (1.25.2) Requirement already satisfied: connected-components-3d<4.0.0,>=3.12.3 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from panoptica->-r requirements.txt (line 1)) (3.12.3) Requirement already satisfied: ruamel.yaml<0.19.0,>=0.18.6 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from panoptica->-r requirements.txt (line 1)) (0.18.6) Requirement already satisfied: scikit-image<0.23.0,>=0.22.0 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from panoptica->-r requirements.txt (line 1)) (0.22.0) Requirement already satisfied: scipy<2.0.0,>=1.7.0 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from panoptica->-r requirements.txt (line 1)) (1.11.2) Requirement already satisfied: nibabel>=3.0 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from auxiliary->-r requirements.txt (line 2)) (5.1.0) Requirement already satisfied: path>=16.10.0 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from auxiliary->-r requirements.txt (line 2)) (17.0.0) Requirement already satisfied: pathlib>=1.0 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from auxiliary->-r requirements.txt (line 2)) (1.0.1) Requirement already satisfied: pillow>=10.0.0 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from auxiliary->-r requirements.txt (line 2)) (10.0.0) Requirement already satisfied: tifffile>=2023.8.25 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from auxiliary->-r requirements.txt (line 2)) (2023.8.30) Requirement already satisfied: markdown-it-py>=2.2.0 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from rich->-r requirements.txt (line 3)) (3.0.0) Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from rich->-r requirements.txt (line 3)) (2.17.2) Requirement already satisfied: mdurl~=0.1 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from markdown-it-py>=2.2.0->rich->-r requirements.txt (line 3)) (0.1.2) Requirement already satisfied: packaging>=17 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from nibabel>=3.0->auxiliary->-r requirements.txt (line 2)) (23.1) Requirement already satisfied: ruamel.yaml.clib>=0.2.7 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from ruamel.yaml<0.19.0,>=0.18.6->panoptica->-r requirements.txt (line 1)) (0.2.8) Requirement already satisfied: networkx>=2.8 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from scikit-image<0.23.0,>=0.22.0->panoptica->-r requirements.txt (line 1)) (3.1) Requirement already satisfied: imageio>=2.27 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from scikit-image<0.23.0,>=0.22.0->panoptica->-r requirements.txt (line 1)) (2.31.3) Requirement already satisfied: lazy_loader>=0.3 in /opt/anaconda3/envs/seg11panoptdev/lib/python3.11/site-packages (from scikit-image<0.23.0,>=0.22.0->panoptica->-r requirements.txt (line 1)) (0.3)
Setup Imports¶
In [2]:
from auxiliary.nifti.io import read_nifti
from rich import print as pprint
from panoptica import (
InputType,
Panoptica_Evaluator,
ConnectedComponentsInstanceApproximator,
NaiveThresholdMatching,
)
Load Data¶
To demonstrate we use a reference and predicition of spine a segmentation without instances.
In [3]:
ref_masks = read_nifti("./spine_seg/semantic/ref.nii.gz")
pred_masks = read_nifti("./spine_seg/semantic/pred.nii.gz")
To use your own data please replace the example data with your own data.
In ordner to successfully load your data please use NIFTI files and the following file designation within the "semantic" folder:
panoptica/spine_seg/semantic/
- Reference data ("ref.nii.gz")
- Prediction data ("pred.nii.gz")
Run Evaluation¶
In [4]:
evaluator = Panoptica_Evaluator(
expected_input=InputType.SEMANTIC,
instance_approximator=ConnectedComponentsInstanceApproximator(),
instance_matcher=NaiveThresholdMatching(),
)
Inspect Results¶
The results object allows access to individual metrics and provides helper methods for further processing
In [5]:
# print all results
result, intermediate_steps_data = evaluator.evaluate(
pred_masks, ref_masks, verbose=False
)["ungrouped"]
print(result)
────────────────────────────────────────── Thank you for using panoptica ──────────────────────────────────────────
Please support our development by citing
https://github.com/BrainLesion/panoptica#citation -- Thank you!
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────
+++ MATCHING +++ Number of instances in reference (num_ref_instances): 87 Number of instances in prediction (num_pred_instances): 89 True Positives (tp): 73 False Positives (fp): 16 False Negatives (fn): 14 Recognition Quality / F1-Score (rq): 0.8295454545454546 +++ GLOBAL +++ Global Binary Dice (global_bin_dsc): 0.9731641527805414 +++ INSTANCE +++ Segmentation Quality IoU (sq): 0.7940127477906024 +- 0.11547745015679488 Panoptic Quality IoU (pq): 0.6586696657808406 Segmentation Quality Dsc (sq_dsc): 0.8802182546605446 +- 0.07728416427007168 Panoptic Quality Dsc (pq_dsc): 0.7301810521615881 Segmentation Quality ASSD (sq_assd): 0.2057371092494465 +- 0.1398348236766068 Segmentation Quality Relative Volume Difference (sq_rvd): 0.011340219860617232 +- 0.1217805112447998
In [6]:
# get specific metric, e.g. pq
pprint(f"{result.pq=}")
result.pq=0.6586696657808406
In [7]:
# get dict for further processing, e.g. for pandas
pprint("results dict: ", result.to_dict())
results dict: { 'num_ref_instances': 87, 'num_pred_instances': 89, 'tp': 73, 'fp': 16, 'fn': 14, 'prec': 0.8202247191011236, 'rec': 0.8390804597701149, 'rq': 0.8295454545454546, 'sq': 0.7940127477906024, 'sq_std': 0.11547745015679488, 'pq': 0.6586696657808406, 'sq_dsc': 0.8802182546605446, 'sq_dsc_std': 0.07728416427007168, 'pq_dsc': 0.7301810521615881, 'sq_assd': 0.2057371092494465, 'sq_assd_std': 0.1398348236766068, 'sq_rvd': 0.011340219860617232, 'sq_rvd_std': 0.1217805112447998, 'global_bin_dsc': 0.9731641527805414 }
In [8]:
# To inspect different phases, just use the returned intermediate_steps_data object
import numpy as np
intermediate_steps_data.original_prediction_arr # yields input prediction array
intermediate_steps_data.original_reference_arr # yields input reference array
intermediate_steps_data.prediction_arr(
InputType.MATCHED_INSTANCE
) # yields prediction array after instances have been matched
intermediate_steps_data.reference_arr(
InputType.MATCHED_INSTANCE
) # yields reference array after instances have been matched
# This works with all InputType
for i in InputType:
print(i)
pred = intermediate_steps_data.prediction_arr(i)
ref = intermediate_steps_data.reference_arr(i)
print("Prediction array shape =", pred.shape, "unique_values=", np.unique(pred))
print("Reference array shape =", ref.shape, "unique_values=", np.unique(ref))
print()
InputType.SEMANTIC Prediction array shape = (170, 512, 17) unique_values= [ 0 26 41 42 43 44 45 46 47 48 49 60 61 62 100] Reference array shape = (170, 512, 17) unique_values= [ 0 26 41 42 43 44 45 46 47 48 49 60 61 62 100] InputType.UNMATCHED_INSTANCE Prediction array shape = (170, 512, 17) unique_values= [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89] Reference array shape = (170, 512, 17) unique_values= [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87] InputType.MATCHED_INSTANCE Prediction array shape = (170, 512, 17) unique_values= [ 0 1 2 4 5 6 9 10 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 32 33 34 35 36 37 38 39 41 42 43 44 45 49 50 51 52 54 55 56 57 58 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103] Reference array shape = (170, 512, 17) unique_values= [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87]