As of v21.10.01, cuCIM supports Aperio SVS Format(.svs).
!pip install cucim
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: cucim in /home/gbae/repo/cucim/python/cucim/src (21.12.0)
Requirement already satisfied: click in /home/gbae/.pyenv/versions/3.8.10/envs/cucim-3.8/lib/python3.8/site-packages (from cucim) (8.0.1)
Requirement already satisfied: numpy in /home/gbae/.pyenv/versions/3.8.10/envs/cucim-3.8/lib/python3.8/site-packages (from cucim) (1.21.2)
WARNING: You are using pip version 21.2.4; however, version 21.3.1 is available.
You should consider upgrading via the '/home/gbae/.pyenv/versions/3.8.10/envs/cucim-3.8/bin/python3.8 -m pip install --upgrade pip' command.
!pip install wget matplotlib
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: wget in /home/gbae/.pyenv/versions/3.8.10/envs/cucim-3.8/lib/python3.8/site-packages (3.2)
Requirement already satisfied: matplotlib in /home/gbae/.pyenv/versions/3.8.10/envs/cucim-3.8/lib/python3.8/site-packages (3.4.2)
Requirement already satisfied: cycler>=0.10 in /home/gbae/.pyenv/versions/3.8.10/envs/cucim-3.8/lib/python3.8/site-packages (from matplotlib) (0.10.0)
Requirement already satisfied: pillow>=6.2.0 in /home/gbae/.pyenv/versions/3.8.10/envs/cucim-3.8/lib/python3.8/site-packages (from matplotlib) (8.2.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /home/gbae/.pyenv/versions/3.8.10/envs/cucim-3.8/lib/python3.8/site-packages (from matplotlib) (1.3.1)
Requirement already satisfied: python-dateutil>=2.7 in /home/gbae/.pyenv/versions/3.8.10/envs/cucim-3.8/lib/python3.8/site-packages (from matplotlib) (2.8.1)
Requirement already satisfied: pyparsing>=2.2.1 in /home/gbae/.pyenv/versions/3.8.10/envs/cucim-3.8/lib/python3.8/site-packages (from matplotlib) (2.4.7)
Requirement already satisfied: numpy>=1.16 in /home/gbae/.pyenv/versions/3.8.10/envs/cucim-3.8/lib/python3.8/site-packages (from matplotlib) (1.21.2)
Requirement already satisfied: six in /home/gbae/.pyenv/versions/3.8.10/envs/cucim-3.8/lib/python3.8/site-packages (from cycler>=0.10->matplotlib) (1.16.0)
WARNING: You are using pip version 21.2.4; however, version 21.3.1 is available.
You should consider upgrading via the '/home/gbae/.pyenv/versions/3.8.10/envs/cucim-3.8/bin/python3.8 -m pip install --upgrade pip' command.
We can download sample SVS files from OpenSlide Test Data which is available under CC0-1.0 LICENSE.
DATA_PATH_LIST = []
import json
import wget
from pathlib import Path
data_url = "https://openslide.cs.cmu.edu/download/openslide-testdata"
if not Path("index.json").exists():
wget.download(f"{data_url}/index.json")
with open("index.json") as fp:
data_index = json.load(fp)
Path("Aperio").mkdir(parents=True, exist_ok=True)
for file_path in data_index:
if file_path.startswith("Aperio/"):
DATA_PATH_LIST.append(file_path)
if not Path(file_path).exists():
file_info = data_index[file_path]
print(f"# Downloading {file_path} ({file_info['description']}) ...")
wget.download(f"{data_url}/{file_path}", out=file_path)
print()
# Downloading Aperio/CMU-1-JP2K-33005.svs (Export of CMU-1.svs, brightfield, JPEG 2000, RGB) ... 100% [......................................................................] 132565343 / 132565343 # Downloading Aperio/CMU-1-Small-Region.svs (Exported region from CMU-1.svs, brightfield, JPEG, small enough to have a single pyramid level) ... 100% [..........................................................................] 1938955 / 1938955 # Downloading Aperio/CMU-1.svs (Brightfield, JPEG) ... 100% [......................................................................] 177552579 / 177552579 # Downloading Aperio/CMU-2.svs (Brightfield, JPEG) ... 100% [......................................................................] 390750635 / 390750635 # Downloading Aperio/CMU-3.svs (Brightfield, JPEG) ... 100% [......................................................................] 253815723 / 253815723 # Downloading Aperio/JP2K-33003-1.svs (Aorta tissue, brightfield, JPEG 2000, YCbCr) ... 100% [........................................................................] 63847265 / 63847265 # Downloading Aperio/JP2K-33003-2.svs (Heart tissue, brightfield, JPEG 2000, YCbCr) ... 100% [......................................................................] 289250433 / 289250433
Aperio SVS-specific metadata is available at aperio
of metadata
field of the CuImage
object.
from cucim import CuImage
import pprint
pp = pprint.PrettyPrinter(indent=2, compact=True)
for file_name in DATA_PATH_LIST:
img = CuImage(file_name)
print("#", file_name)
metadata = img.metadata
pp.pprint(img.metadata)
# Aperio/CMU-1-JP2K-33005.svs { 'aperio': { 'AppMag': '20', 'Date': '12/29/09', 'Filename': 'CMU-1', 'Filtered': '5', 'Focus Offset': '0.000000', 'Header': 'Aperio Image Library v11.2.1 \r\n' '46000x32914 [0,0 46000x32893] (240x240) J2K/KDU ' 'Q=30;CMU-1;Aperio Image Library v10.0.51\r\n' '46920x33014 [0,100 46000x32914] (256x256) JPEG/RGB ' 'Q=30', 'ImageID': '1004486', 'Left': '25.691574', 'LineAreaXOffset': '0.019265', 'LineAreaYOffset': '-0.000313', 'LineCameraSkew': '-0.000424', 'MPP': '0.4990', 'OriginalHeight': '32914', 'OriginalWidth': '46920', 'Originalheight': '33014', 'Parmset': 'USM Filter', 'ScanScope ID': 'CPAPERIOCS', 'StripeWidth': '2040', 'Time': '09:59:15', 'Top': '23.449873', 'User': 'b414003d-95c6-48b0-9369-8010ed517ba7'}, 'cucim': { 'associated_images': ['label', 'macro', 'thumbnail'], 'channel_names': ['R', 'G', 'B'], 'coord_sys': 'LPS', 'dims': 'YXC', 'direction': [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]], 'dtype': {'bits': 8, 'code': 1, 'lanes': 1}, 'ndim': 3, 'origin': [0.0, 0.0, 0.0], 'path': 'Aperio/CMU-1-JP2K-33005.svs', 'resolutions': { 'level_count': 3, 'level_dimensions': [ [46000, 32893], [11500, 8223], [2875, 2055]], 'level_downsamples': [ 1.0, 4.00006103515625, 16.003162384033203], 'level_tile_sizes': [ [240, 240], [240, 240], [240, 240]]}, 'shape': [32893, 46000, 3], 'spacing': [1.0, 1.0, 1.0], 'spacing_units': ['micrometer', 'micrometer', 'color']}, 'tiff': {'model': '', 'software': ''}} # Aperio/CMU-1-Small-Region.svs { 'aperio': { 'AppMag': '20', 'Date': '12/29/09', 'Filename': 'CMU-1', 'Filtered': '5', 'Focus Offset': '0.000000', 'Header': 'Aperio Image Library v11.2.1 \r\n' '46000x32914 [42673,5576 2220x2967] (240x240) JPEG/RGB ' 'Q=30;Aperio Image Library v10.0.51\r\n' '46920x33014 [0,100 46000x32914] (256x256) JPEG/RGB ' 'Q=30', 'ImageID': '1004486', 'Left': '25.691574', 'LineAreaXOffset': '0.019265', 'LineAreaYOffset': '-0.000313', 'LineCameraSkew': '-0.000424', 'MPP': '0.4990', 'OriginalHeight': '32914', 'OriginalWidth': '46920', 'Originalheight': '33014', 'Parmset': 'USM Filter', 'ScanScope ID': 'CPAPERIOCS', 'StripeWidth': '2040', 'Time': '09:59:15', 'Top': '23.449873', 'User': 'b414003d-95c6-48b0-9369-8010ed517ba7'}, 'cucim': { 'associated_images': ['label', 'macro', 'thumbnail'], 'channel_names': ['R', 'G', 'B'], 'coord_sys': 'LPS', 'dims': 'YXC', 'direction': [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]], 'dtype': {'bits': 8, 'code': 1, 'lanes': 1}, 'ndim': 3, 'origin': [0.0, 0.0, 0.0], 'path': 'Aperio/CMU-1-Small-Region.svs', 'resolutions': { 'level_count': 1, 'level_dimensions': [[2220, 2967]], 'level_downsamples': [1.0], 'level_tile_sizes': [[240, 240]]}, 'shape': [2967, 2220, 3], 'spacing': [1.0, 1.0, 1.0], 'spacing_units': ['micrometer', 'micrometer', 'color']}, 'tiff': {'model': '', 'software': ''}} # Aperio/CMU-1.svs { 'aperio': { 'AppMag': '20', 'Date': '12/29/09', 'Filename': 'CMU-1', 'Filtered': '5', 'Focus Offset': '0.000000', 'Header': 'Aperio Image Library v10.0.51\r\n' '46920x33014 [0,100 46000x32914] (256x256) JPEG/RGB ' 'Q=30', 'ICC Profile': 'ScanScope v1', 'ImageID': '1004486', 'Left': '25.691574', 'LineAreaXOffset': '0.019265', 'LineAreaYOffset': '-0.000313', 'LineCameraSkew': '-0.000424', 'MPP': '0.4990', 'OriginalWidth': '46920', 'Originalheight': '33014', 'Parmset': 'USM Filter', 'ScanScope ID': 'CPAPERIOCS', 'StripeWidth': '2040', 'Time': '09:59:15', 'Top': '23.449873', 'User': 'b414003d-95c6-48b0-9369-8010ed517ba7'}, 'cucim': { 'associated_images': ['label', 'macro', 'thumbnail'], 'channel_names': ['R', 'G', 'B'], 'coord_sys': 'LPS', 'dims': 'YXC', 'direction': [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]], 'dtype': {'bits': 8, 'code': 1, 'lanes': 1}, 'ndim': 3, 'origin': [0.0, 0.0, 0.0], 'path': 'Aperio/CMU-1.svs', 'resolutions': { 'level_count': 3, 'level_dimensions': [ [46000, 32914], [11500, 8228], [2875, 2057]], 'level_downsamples': [ 1.0, 4.000121593475342, 16.000486373901367], 'level_tile_sizes': [ [256, 256], [256, 256], [256, 256]]}, 'shape': [32914, 46000, 3], 'spacing': [1.0, 1.0, 1.0], 'spacing_units': ['micrometer', 'micrometer', 'color']}, 'tiff': {'model': '', 'software': ''}} # Aperio/CMU-2.svs { 'aperio': { 'AppMag': '20', 'Date': '12/29/09', 'Filename': 'CMU-2', 'Filtered': '5', 'Focus Offset': '0.000000', 'Header': 'Aperio Image Library v10.0.51\r\n' '79560x30562 [0,100 78000x30462] (256x256) JPEG/RGB ' 'Q=30', 'ICC Profile': 'ScanScope v1', 'ImageID': '1004487', 'Left': '27.409658', 'LineAreaXOffset': '0.019265', 'LineAreaYOffset': '-0.000313', 'LineCameraSkew': '-0.000424', 'MPP': '0.4990', 'OriginalWidth': '79560', 'Originalheight': '30562', 'Parmset': 'USM Filter', 'ScanScope ID': 'CPAPERIOCS', 'StripeWidth': '2040', 'Time': '10:02:42', 'Top': '20.522137', 'User': 'b414003d-95c6-48b0-9369-8010ed517ba7'}, 'cucim': { 'associated_images': ['label', 'macro', 'thumbnail'], 'channel_names': ['R', 'G', 'B'], 'coord_sys': 'LPS', 'dims': 'YXC', 'direction': [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]], 'dtype': {'bits': 8, 'code': 1, 'lanes': 1}, 'ndim': 3, 'origin': [0.0, 0.0, 0.0], 'path': 'Aperio/CMU-2.svs', 'resolutions': { 'level_count': 4, 'level_dimensions': [ [78000, 30462], [19500, 7615], [4875, 1903], [2437, 951]], 'level_downsamples': [ 1.0, 4.000131607055664, 16.003677368164062, 32.01905822753906], 'level_tile_sizes': [ [256, 256], [256, 256], [256, 256], [256, 256]]}, 'shape': [30462, 78000, 3], 'spacing': [1.0, 1.0, 1.0], 'spacing_units': ['micrometer', 'micrometer', 'color']}, 'tiff': {'model': '', 'software': ''}} # Aperio/CMU-3.svs { 'aperio': { 'AppMag': '20', 'Date': '12/29/09', 'Filename': 'CMU-3', 'Filtered': '5', 'Focus Offset': '0.000000', 'Header': 'Aperio Image Library v10.0.51\r\n' '67320x45502 [0,100 66000x45402] (256x256) JPEG/RGB ' 'Q=30', 'ICC Profile': 'ScanScope v1', 'ImageID': '1004488', 'Left': '22.445614', 'LineAreaXOffset': '0.019265', 'LineAreaYOffset': '-0.000313', 'LineCameraSkew': '-0.000424', 'MPP': '0.4990', 'OriginalWidth': '67320', 'Originalheight': '45502', 'Parmset': 'USM Filter', 'ScanScope ID': 'CPAPERIOCS', 'StripeWidth': '2040', 'Time': '10:07:20', 'Top': '24.084003', 'User': 'b414003d-95c6-48b0-9369-8010ed517ba7'}, 'cucim': { 'associated_images': ['label', 'macro', 'thumbnail'], 'channel_names': ['R', 'G', 'B'], 'coord_sys': 'LPS', 'dims': 'YXC', 'direction': [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]], 'dtype': {'bits': 8, 'code': 1, 'lanes': 1}, 'ndim': 3, 'origin': [0.0, 0.0, 0.0], 'path': 'Aperio/CMU-3.svs', 'resolutions': { 'level_count': 4, 'level_dimensions': [ [66000, 45402], [16500, 11350], [4125, 2837], [2062, 1418]], 'level_downsamples': [ 1.0, 4.000088214874268, 16.00176239013672, 32.01304626464844], 'level_tile_sizes': [ [256, 256], [256, 256], [256, 256], [256, 256]]}, 'shape': [45402, 66000, 3], 'spacing': [1.0, 1.0, 1.0], 'spacing_units': ['micrometer', 'micrometer', 'color']}, 'tiff': {'model': '', 'software': ''}} # Aperio/JP2K-33003-1.svs { 'aperio': { 'AppMag': '40', 'DSR ID': 'homer', 'Date': '07/16/09', 'Filename': '6797', 'Filtered': '3', 'Focus Offset': '-0.001000', 'Header': 'Aperio Image Library v10.0.50\r\n' '16000x17597 [0,100 15374x17497] (256x256) J2K/YUV16 ' 'Q=70', 'ICC Profile': 'ScanScope v1', 'ImageID': '6797', 'Left': '39.010742', 'LineAreaXOffset': '0.000000', 'LineAreaYOffset': '0.000000', 'LineCameraSkew': '-0.003035', 'MPP': '0.2498', 'OriginalWidth': '16000', 'Originalheight': '17597', 'ScanScope ID': 'SS1283', 'StripeWidth': '1000', 'Time': '18:15:06', 'Title': 'univ missouri 07.15.09', 'Top': '14.299895', 'User': '93d70f65-3b32-4072-ba6a-bd6785a781be'}, 'cucim': { 'associated_images': ['label', 'macro', 'thumbnail'], 'channel_names': ['R', 'G', 'B'], 'coord_sys': 'LPS', 'dims': 'YXC', 'direction': [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]], 'dtype': {'bits': 8, 'code': 1, 'lanes': 1}, 'ndim': 3, 'origin': [0.0, 0.0, 0.0], 'path': 'Aperio/JP2K-33003-1.svs', 'resolutions': { 'level_count': 3, 'level_dimensions': [ [15374, 17497], [3843, 4374], [1921, 2187]], 'level_downsamples': [ 1.0, 4.0003743171691895, 8.001790046691895], 'level_tile_sizes': [ [256, 256], [256, 256], [256, 256]]}, 'shape': [17497, 15374, 3], 'spacing': [1.0, 1.0, 1.0], 'spacing_units': ['micrometer', 'micrometer', 'color']}, 'tiff': {'model': '', 'software': ''}} # Aperio/JP2K-33003-2.svs { 'aperio': { 'AppMag': '40', 'DSR ID': 'homer', 'Date': '07/16/09', 'Filename': '6811', 'Filtered': '3', 'Focus Offset': '-0.001000', 'Header': 'Aperio Image Library v10.0.50\r\n' '34000x47176 [0,100 32671x47076] (256x256) J2K/YUV16 ' 'Q=70', 'ICC Profile': 'ScanScope v1', 'ImageID': '6811', 'Left': '44.815720', 'LineAreaXOffset': '0.000000', 'LineAreaYOffset': '0.000000', 'LineCameraSkew': '-0.003035', 'MPP': '0.2498', 'OriginalWidth': '34000', 'Originalheight': '47176', 'ScanScope ID': 'SS1283', 'StripeWidth': '1000', 'Time': '21:59:00', 'Title': 'univ missouri 07.15.09', 'Top': '18.114859', 'User': '93d70f65-3b32-4072-ba6a-bd6785a781be'}, 'cucim': { 'associated_images': ['label', 'macro', 'thumbnail'], 'channel_names': ['R', 'G', 'B'], 'coord_sys': 'LPS', 'dims': 'YXC', 'direction': [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]], 'dtype': {'bits': 8, 'code': 1, 'lanes': 1}, 'ndim': 3, 'origin': [0.0, 0.0, 0.0], 'path': 'Aperio/JP2K-33003-2.svs', 'resolutions': { 'level_count': 3, 'level_dimensions': [ [32671, 47076], [8167, 11769], [2041, 2942]], 'level_downsamples': [ 1.0, 4.000183582305908, 16.00435447692871], 'level_tile_sizes': [ [256, 256], [256, 256], [256, 256]]}, 'shape': [47076, 32671, 3], 'spacing': [1.0, 1.0, 1.0], 'spacing_units': ['micrometer', 'micrometer', 'color']}, 'tiff': {'model': '', 'software': ''}}
associated_images
field has a set of available associated images such as 'label', 'macro', and 'thumbnail' images.
img.associated_images
{'label', 'macro', 'thumbnail'}
from matplotlib import pyplot as plt
def visualize(image, downsample=1):
dpi = 80.0 * downsample
height, width, _ = image.shape
plt.figure(figsize=(width / dpi, height / dpi))
plt.axis('off')
plt.imshow(image)
plt.close('all')
import numpy as np
from cucim import CuImage
for file_name in DATA_PATH_LIST:
img = CuImage(file_name)
metadata = img.metadata
level_count = metadata["cucim"]["resolutions"]["level_count"]
small_img = img.read_region(level=level_count - 1) # read whole image at the lowest resolution level
label_img = img.associated_image("label")
macro_img = img.associated_image("macro")
thumbnail_img = img.associated_image("thumbnail")
visualize(np.asarray(small_img), 5)
visualize(np.asarray(label_img), 2)
visualize(np.asarray(macro_img), 3)
visualize(np.asarray(thumbnail_img), 4)
/tmp/ipykernel_8881/1677916248.py:6: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`). plt.figure(figsize=(width / dpi, height / dpi))
Like the experiments in Using Cache, we have compared performance against OpenSlide.
For the cache memory size(capacity) setting, we have used a similar approach with rasterio (5% of available system memory).
Benchmarked loading Aperio SVS format with OpenSlide.
per_process
, JPEG-compressed SVS file)¶per_process
, JPEG2000 RGB-compressed SVS file)¶per_process
, JPEG2000 YCbCr-compressed SVS file)¶The detailed data is available here.
Please see https://github.com/rapidsai/cucim/blob/branch-21.12/experiments/Supporting_Aperio_SVS_Format/benchmark.py to check out the code used for the experiment.