Multi-scale Tissue Segmentation on Canine Cutaneous Tumors¶
In [ ]:
#!pip install EXCAT-Sync
#!pip install requests_toolbelt
Collecting EXCAT-Sync
Downloading https://files.pythonhosted.org/packages/21/22/88ea258b98769879ae4a4d717ee02a6cd622bfc81fcd29434d79efa7c355/EXCAT_Sync-0.0.17-py3-none-any.whl (115kB)
|████████████████████████████████| 122kB 7.6MB/s
Installing collected packages: EXCAT-Sync
Successfully installed EXCAT-Sync-0.0.17
Collecting requests_toolbelt
Downloading https://files.pythonhosted.org/packages/60/ef/7681134338fc097acef8d9b2f8abe0458e4d87559c689a8c306d0957ece5/requests_toolbelt-0.9.1-py2.py3-none-any.whl (54kB)
|████████████████████████████████| 61kB 4.6MB/s
Requirement already satisfied: requests<3.0.0,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from requests_toolbelt) (2.23.0)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests<3.0.0,>=2.0.1->requests_toolbelt) (2.10)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests<3.0.0,>=2.0.1->requests_toolbelt) (2020.6.20)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests<3.0.0,>=2.0.1->requests_toolbelt) (1.24.3)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests<3.0.0,>=2.0.1->requests_toolbelt) (3.0.4)
Installing collected packages: requests-toolbelt
Successfully installed requests-toolbelt-0.9.1
Install lib for OpenSlide¶
TODO: I would recomment using Conda with FastAi https://github.com/fastai/fastai#conda-install
TODO: Install OpenSlide for Windows on your LAB computer https://openslide.org/download/
In [ ]:
#!apt-get install python3-openslide
Reading package lists... Done Building dependency tree Reading state information... Done The following package was automatically installed and is no longer required: libnvidia-common-440 Use 'apt autoremove' to remove it. The following additional packages will be installed: javascript-common libjs-jquery libopenslide0 python-asn1crypto python-blinker python-cffi-backend python-click python-colorama python-cryptography python-enum34 python-flask python-idna python-ipaddress python-itsdangerous python-jinja2 python-markupsafe python-openslide-examples python-openssl python-pkg-resources python-pyinotify python-simplejson python-six python-werkzeug python3-olefile python3-pil Suggested packages: apache2 | lighttpd | httpd python-blinker-doc python-cryptography-doc python-cryptography-vectors python-enum34-doc python-flask-doc python-jinja2-doc python-openssl-doc python-openssl-dbg python-setuptools python-pyinotify-doc ipython python-genshi python-lxml python-greenlet python-redis python-pylibmc | python-memcache python-termcolor python-watchdog python-werkzeug-doc python-pil-doc python3-pil-dbg The following NEW packages will be installed: javascript-common libjs-jquery libopenslide0 python-asn1crypto python-blinker python-cffi-backend python-click python-colorama python-cryptography python-enum34 python-flask python-idna python-ipaddress python-itsdangerous python-jinja2 python-markupsafe python-openslide-examples python-openssl python-pkg-resources python-pyinotify python-simplejson python-six python-werkzeug python3-olefile python3-openslide python3-pil 0 upgraded, 26 newly installed, 0 to remove and 35 not upgraded. Need to get 1,942 kB of archives. After this operation, 9,494 kB of additional disk space will be used. Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 javascript-common all 11 [6,066 B] Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 libjs-jquery all 3.2.1-1 [152 kB] Get:3 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libopenslide0 amd64 3.4.1+dfsg-2 [79.8 kB] Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-asn1crypto all 0.24.0-1 [72.7 kB] Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-blinker all 1.4+dfsg1-0.1 [13.0 kB] Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-cffi-backend amd64 1.11.5-1 [63.4 kB] Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-colorama all 0.3.7-1 [22.6 kB] Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-click all 6.7-3 [56.4 kB] Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-enum34 all 1.1.6-2 [34.8 kB] Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-idna all 2.6-1 [32.4 kB] Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-ipaddress all 1.0.17-1 [18.2 kB] Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-six all 1.11.0-2 [11.3 kB] Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python-cryptography amd64 2.1.4-1ubuntu1.3 [221 kB] Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-itsdangerous all 0.24+dfsg1-2 [11.9 kB] Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-markupsafe amd64 1.0-1build1 [13.0 kB] Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python-jinja2 all 2.10-1ubuntu0.18.04.1 [94.8 kB] Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-werkzeug all 0.14.1+dfsg1-1 [174 kB] Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python-flask all 0.12.2-3ubuntu0.1 [62.4 kB] Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] Get:20 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-openslide amd64 1.1.1-2ubuntu4 [16.1 kB] Get:21 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-openslide-examples all 1.1.1-2ubuntu4 [168 kB] Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-openssl all 17.5.0-1ubuntu1 [41.3 kB] Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pkg-resources all 39.0.1-2 [128 kB] Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pyinotify all 0.9.6-1 [24.6 kB] Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-simplejson amd64 3.13.2-1 [61.2 kB] Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] Fetched 1,942 kB in 0s (14.2 MB/s) Selecting previously unselected package javascript-common. (Reading database ... 144487 files and directories currently installed.) Preparing to unpack .../00-javascript-common_11_all.deb ... Unpacking javascript-common (11) ... Selecting previously unselected package libjs-jquery. Preparing to unpack .../01-libjs-jquery_3.2.1-1_all.deb ... Unpacking libjs-jquery (3.2.1-1) ... Selecting previously unselected package libopenslide0. Preparing to unpack .../02-libopenslide0_3.4.1+dfsg-2_amd64.deb ... Unpacking libopenslide0 (3.4.1+dfsg-2) ... Selecting previously unselected package python-asn1crypto. Preparing to unpack .../03-python-asn1crypto_0.24.0-1_all.deb ... Unpacking python-asn1crypto (0.24.0-1) ... Selecting previously unselected package python-blinker. Preparing to unpack .../04-python-blinker_1.4+dfsg1-0.1_all.deb ... Unpacking python-blinker (1.4+dfsg1-0.1) ... Selecting previously unselected package python-cffi-backend. Preparing to unpack .../05-python-cffi-backend_1.11.5-1_amd64.deb ... Unpacking python-cffi-backend (1.11.5-1) ... Selecting previously unselected package python-colorama. Preparing to unpack .../06-python-colorama_0.3.7-1_all.deb ... Unpacking python-colorama (0.3.7-1) ... Selecting previously unselected package python-click. Preparing to unpack .../07-python-click_6.7-3_all.deb ... Unpacking python-click (6.7-3) ... Selecting previously unselected package python-enum34. Preparing to unpack .../08-python-enum34_1.1.6-2_all.deb ... Unpacking python-enum34 (1.1.6-2) ... Selecting previously unselected package python-idna. Preparing to unpack .../09-python-idna_2.6-1_all.deb ... Unpacking python-idna (2.6-1) ... Selecting previously unselected package python-ipaddress. Preparing to unpack .../10-python-ipaddress_1.0.17-1_all.deb ... Unpacking python-ipaddress (1.0.17-1) ... Selecting previously unselected package python-six. Preparing to unpack .../11-python-six_1.11.0-2_all.deb ... Unpacking python-six (1.11.0-2) ... Selecting previously unselected package python-cryptography. Preparing to unpack .../12-python-cryptography_2.1.4-1ubuntu1.3_amd64.deb ... Unpacking python-cryptography (2.1.4-1ubuntu1.3) ... Selecting previously unselected package python-itsdangerous. Preparing to unpack .../13-python-itsdangerous_0.24+dfsg1-2_all.deb ... Unpacking python-itsdangerous (0.24+dfsg1-2) ... Selecting previously unselected package python-markupsafe. Preparing to unpack .../14-python-markupsafe_1.0-1build1_amd64.deb ... Unpacking python-markupsafe (1.0-1build1) ... Selecting previously unselected package python-jinja2. Preparing to unpack .../15-python-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... Unpacking python-jinja2 (2.10-1ubuntu0.18.04.1) ... Selecting previously unselected package python-werkzeug. Preparing to unpack .../16-python-werkzeug_0.14.1+dfsg1-1_all.deb ... Unpacking python-werkzeug (0.14.1+dfsg1-1) ... Selecting previously unselected package python-flask. Preparing to unpack .../17-python-flask_0.12.2-3ubuntu0.1_all.deb ... Unpacking python-flask (0.12.2-3ubuntu0.1) ... Selecting previously unselected package python3-pil:amd64. Preparing to unpack .../18-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... Selecting previously unselected package python3-openslide. Preparing to unpack .../19-python3-openslide_1.1.1-2ubuntu4_amd64.deb ... Unpacking python3-openslide (1.1.1-2ubuntu4) ... Selecting previously unselected package python-openslide-examples. Preparing to unpack .../20-python-openslide-examples_1.1.1-2ubuntu4_all.deb ... Unpacking python-openslide-examples (1.1.1-2ubuntu4) ... Selecting previously unselected package python-openssl. Preparing to unpack .../21-python-openssl_17.5.0-1ubuntu1_all.deb ... Unpacking python-openssl (17.5.0-1ubuntu1) ... Selecting previously unselected package python-pkg-resources. Preparing to unpack .../22-python-pkg-resources_39.0.1-2_all.deb ... Unpacking python-pkg-resources (39.0.1-2) ... Selecting previously unselected package python-pyinotify. Preparing to unpack .../23-python-pyinotify_0.9.6-1_all.deb ... Unpacking python-pyinotify (0.9.6-1) ... Selecting previously unselected package python-simplejson. Preparing to unpack .../24-python-simplejson_3.13.2-1_amd64.deb ... Unpacking python-simplejson (3.13.2-1) ... Selecting previously unselected package python3-olefile. Preparing to unpack .../25-python3-olefile_0.45.1-1_all.deb ... Unpacking python3-olefile (0.45.1-1) ... Setting up python-idna (2.6-1) ... Setting up python-simplejson (3.13.2-1) ... Setting up libjs-jquery (3.2.1-1) ... Setting up python-asn1crypto (0.24.0-1) ... Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... Setting up python-blinker (1.4+dfsg1-0.1) ... Setting up python3-olefile (0.45.1-1) ... Setting up python-colorama (0.3.7-1) ... Setting up python-pkg-resources (39.0.1-2) ... Setting up python-markupsafe (1.0-1build1) ... Setting up python-werkzeug (0.14.1+dfsg1-1) ... Setting up python-pyinotify (0.9.6-1) ... Setting up python-cffi-backend (1.11.5-1) ... Setting up python-six (1.11.0-2) ... Setting up libopenslide0 (3.4.1+dfsg-2) ... Setting up python-enum34 (1.1.6-2) ... Setting up javascript-common (11) ... Setting up python-itsdangerous (0.24+dfsg1-2) ... Setting up python-ipaddress (1.0.17-1) ... Setting up python-jinja2 (2.10-1ubuntu0.18.04.1) ... Setting up python-click (6.7-3) ... Setting up python3-openslide (1.1.1-2ubuntu4) ... Setting up python-cryptography (2.1.4-1ubuntu1.3) ... Setting up python-flask (0.12.2-3ubuntu0.1) ... Setting up python-openssl (17.5.0-1ubuntu1) ... Setting up python-openslide-examples (1.1.1-2ubuntu4) ... Processing triggers for libc-bin (2.27-3ubuntu1) ... /sbin/ldconfig.real: /usr/local/lib/python3.6/dist-packages/ideep4py/lib/libmkldnn.so.0 is not a symbolic link
Import packages¶
In [ ]:
%reload_ext autoreload
%autoreload 2
%matplotlib inline
In [ ]:
import numpy as np
from pathlib import Path
import openslide
from tqdm import tqdm
import cv2
import random
from random import randint
from tqdm import tqdm
import matplotlib
In [ ]:
from fastai import *
from fastai.vision import *
from fastai.callbacks import *
from fastai.data_block import *
from fastai.vision.data import SegmentationProcessor
Confirm that you have a GPU!¶
=== Hardware ===
nvidia gpus : 1
torch devices : 1
- gpu0 : 16280MB | Tesla P100-PCIE-16GB
In [ ]:
from fastai.utils.collect_env import show_install
show_install()
```text === Software === python : 3.6.9 fastai : 1.0.61 fastprogress : 0.2.5 torch : 1.6.0+cu101 nvidia driver : 418.67 torch cuda : 10.1 / is available torch cudnn : 7603 / is enabled === Hardware === nvidia gpus : 1 torch devices : 1 - gpu0 : 15079MB | Tesla T4 === Environment === platform : Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic distro : #1 SMP Thu Jul 23 08:00:38 PDT 2020 conda env : Unknown python : /usr/bin/python3 sys.path : /env/python /usr/lib/python36.zip /usr/lib/python3.6 /usr/lib/python3.6/lib-dynload /usr/local/lib/python3.6/dist-packages /usr/lib/python3/dist-packages /usr/local/lib/python3.6/dist-packages/IPython/extensions /root/.ipython ``` Please make sure to include opening/closing ``` when you paste into forums/github to make the reports appear formatted as code sections. Optional package(s) to enhance the diagnostics can be installed with: pip install distro Once installed, re-run this utility to get the additional information
Slide Container¶
In [ ]:
class SlideContainer():
def __init__(self, file: Path,
y,
level: int = 0,
width: int = 256, height: int = 256,
level_range:list=[0, 1, 2, 3],
sample_func:callable=None):
self.labels, self.coordinates = y
self.file = file
self.slide = openslide.open_slide(str(file))
self.width = width
self.height = height
self.down_factor = self.slide.level_downsamples[level]
if level is None:
level = self.slide.level_count - 1
self._level = level
# remove levels that are not supported!
self.level_range = [l for l in level_range if l < self.slide.level_count]
self.sample_func = sample_func
@property
def level(self):
return self.level
@level.setter
def level(self, value):
self.down_factor = self.slide.level_downsamples[value]
self._level = value
@property
def shape(self):
return (self.width, self.height)
@property
def slide_shape(self):
return self.slide.level_dimensions[self._level]
def get_new_level(self):
return random.choice(self.level_range)
def get_patch(self, x: int = 0, y: int = 0):
return np.array(self.slide.read_region(location=(int(x * self.down_factor), int(y * self.down_factor)),
level=self._level, size=(self.width, self.height)))[:, :, :3]
def get_y_patch(self, x: int = 0, y: int = 0):
y_patch = np.zeros(shape=(self.width, self.height), dtype=np.int8)
coordinates = np.copy(self.coordinates) / self.down_factor
for label, poly in zip(self.labels, coordinates):
poly = poly - (x, y)
cv2.drawContours(y_patch, [poly.reshape((-1, 1, 2)).astype(int)], -1, label, -1)
return y_patch
def get_new_train_coordinates(self):
# use passed sampling method
if callable(self.sample_func):
return self.sample_func(self.y, **{"classes": self.classes, "size": self.shape,
"level_dimensions": self.slide.level_dimensions,
"level": self.level})
# use default sampling method
label = np.random.choice(list(set(self.labels)), 1)[0]
labels_with_coordinates = np.array(list(zip(self.labels, self.coordinates)))
# filter by label
labels_with_coordinates = labels_with_coordinates[labels_with_coordinates[:, 0] == label]
poly = np.random.choice(labels_with_coordinates[:, 1], 1)[0]
xmin, ymin = poly[np.random.choice(len(poly), 1)[0]]
xmin, ymin = xmin / self.down_factor, ymin / self.down_factor
xmin = max(0, int(xmin + randint(-self.width / 2, self.width / 2)))
xmin = min(xmin, self.slide_shape[0] - self.width)
ymin = max(0, int(ymin + randint(-self.height / 2, self.height / 2)))
ymin = min(ymin, self.slide_shape[1] - self.height)
return xmin, ymin
def __str__(self):
return str(self.path)
Some FastAi code to handle WSI and Doku¶
In [ ]:
class SlideLabelList(LabelList):
def __getitem__(self, idxs: Union[int, np.ndarray]) -> 'LabelList':
idxs = try_int(idxs)
if isinstance(idxs, numbers.Integral):
if self.item is None:
slide_container = self.x.items[idxs]
slide_container_y = self.y.items[idxs]
level = slide_container.get_new_level()
slide_container.level = level
slide_container_y.level = level
xmin, ymin = slide_container.get_new_train_coordinates()
x = self.x.get(idxs, xmin, ymin)
y = self.y.get(idxs, xmin, ymin)
else:
x, y = self.item, 0
if self.tfms or self.tfmargs:
x = x.apply_tfms(self.tfms, **self.tfmargs)
if hasattr(self, 'tfms_y') and self.tfm_y and self.item is None:
y = y.apply_tfms(self.tfms_y, **{**self.tfmargs_y, 'do_resolve': False})
if y is None: y = 0
return x, y
else:
return self.new(self.x[idxs], self.y[idxs])
In [ ]:
PreProcessors = Union[PreProcessor, Collection[PreProcessor]]
fastai_types[PreProcessors] = 'PreProcessors'
class SlideItemList(ItemList):
def __init__(self, items:Iterator, path:PathOrStr='.', label_cls:Callable=None, inner_df:Any=None,
processor:PreProcessors=None, x:'ItemList'=None, ignore_empty:bool=False):
self.path = Path(path)
self.num_parts = len(self.path.parts)
self.items,self.x,self.ignore_empty = items,x,ignore_empty
self.sizes = [None] * len(self.items)
if not isinstance(self.items,np.ndarray): self.items = array(self.items, dtype=object)
self.label_cls,self.inner_df,self.processor = ifnone(label_cls,self._label_cls),inner_df,processor
self._label_list,self._split = SlideLabelList,ItemLists
self.copy_new = ['x', 'label_cls', 'path']
def __getitem__(self,idxs: int, x: int=0, y: int=0)->Any:
idxs = try_int(idxs)
if isinstance(idxs, numbers.Integral):
return self.get(idxs, x, y)
else:
return self.get(*idxs)
class SlideImageItemList(SlideItemList):
pass
class SlideSegmentationItemList(SlideImageItemList, ImageList):
def get(self, i, x: int, y: int):
fn = self.items[i]
res = self.open(fn, x, y)
self.sizes[i] = res.size
return res
def open(self, fn: SlideContainer, x: int=0, y: int=0):
patch = fn.get_patch(x, y) / 255.
return Image(pil2tensor(patch, np.float32))
class SlideSegmentationLabelList(ImageList, SlideImageItemList):
"`ItemList` for segmentation masks."
_processor=SegmentationProcessor
def __init__(self, items:Iterator, classes:Collection=None, **kwargs):
super().__init__(items, **kwargs)
self.copy_new.append('classes')
self.classes,self.loss_func = classes,CrossEntropyFlat(axis=1)
def get(self, i, x: int, y: int):
fn = self.items[i]
res = self.open(fn, x, y)
self.sizes[i] = res.size
return res
def open(self, fn: SlideContainer, x: int=0, y: int=0):
patch = fn.get_y_patch(x, y)
return ImageSegment(pil2tensor(patch, np.float32))
def analyze_pred(self, pred, thresh:float=0.5): return pred.argmax(dim=0)[None]
def reconstruct(self, t:Tensor):
return ImageSegment(t)
Download the data from the EXACT Server¶
In [ ]:
from exact_sync.v1.api.annotations_api import AnnotationsApi
from exact_sync.v1.api.images_api import ImagesApi
from exact_sync.v1.api.image_sets_api import ImageSetsApi
from exact_sync.v1.api.annotation_types_api import AnnotationTypesApi
from exact_sync.v1.api.products_api import ProductsApi
from exact_sync.v1.api.teams_api import TeamsApi
from exact_sync.v1.models import ImageSet, Team, Product, AnnotationType, Annotation, AnnotationMediaFile
from exact_sync.v1.rest import ApiException
from exact_sync.v1.configuration import Configuration
from exact_sync.v1.api_client import ApiClient
In [ ]:
patch_size = 256
levels = [0, 1, 2]
target_folder = Path("tissue_slides")
configuration = Configuration()
configuration.username = 'exact'
configuration.password = 'exact'
configuration.host = "http://127.0.0.0:1337"
client = ApiClient(configuration)
image_sets_api = ImageSetsApi(client)
annotations_api = AnnotationsApi(client)
annotation_types_api = AnnotationTypesApi(client)
images_api = ImagesApi(client)
product_api = ProductsApi(client)
In [ ]:
# Name of the data-sets to download images and annotations from
os.makedirs(str(target_folder), exist_ok=True)
In [ ]:
image_sets = image_sets_api.list_image_sets(name__contains='Melanoma')
image_sets
Out[ ]:
{'count': 1,
'next': None,
'previous': None,
'results': [{'creator': 1,
'description': '',
'id': 228,
'images': [3189,
3174,
3175,
3176,
3190,
3417,
3418,
3419,
3420,
3421,
3422,
3423,
3424,
3425,
3426,
3427,
3428,
3429,
3430,
3498],
'location': None,
'main_annotation_type': 300,
'name': 'Melanoma',
'path': 'imagetagger_32_228',
'product_set': [82],
'set_tags': [],
'team': 32}]}
In [ ]:
label_dict = {'Bg':0, 'Bone':1, 'Cartilage':2, 'PNST':3, 'Mast Cell Tumor':4, 'Inflamm/Necrosis':5, 'Melanoma':6, 'Plasmacytoma':7, 'SCC':8, 'Trichoblastoma':9, 'Dermis':10, 'Epidermis':11, 'Subcutis':12}
In [ ]:
annotation_types = {}
container = []
for image_set in image_sets.results:
for product in image_set.product_set:
for anno_type in annotation_types_api.list_annotation_types(product=product).results:
annotation_types[anno_type.id] = anno_type
for image in tqdm(images_api.list_images(image_set=image_set.id, pagination=False).results):
if image.id not in [167, 168]:
continue
coordinates, labels = [], []
image_path = target_folder/image.name
for annotation in annotations_api.list_annotations(image=image.id, pagination=False).results:
annotation.annotation_type = annotation_types[annotation.annotation_type]
# if file not exists download it
if image_path.is_file() == False:
images_api.download_image(id=image.id, target_path=image_path)
vector = []
for i in range(1, (len(annotation.vector) // 2) + 1):
vector.append([annotation.vector['x' + str(i)], annotation.vector['y' + str(i)]])
coordinates.append(np.array(vector))
labels.append(label_dict[annotation.annotation_type.name])
if len(coordinates) > 0:
container.append(SlideContainer(image_path, (labels, coordinates), 0, patch_size, patch_size, level_range=levels))
100%|██████████| 20/20 [10:34<00:00, 31.71s/it]
In [ ]:
train_images = 1000
val_images = 500
Create FastAi data bunch¶
In [ ]:
def get_y_func(x):
return x
# find a good train test split!
train_files = list(np.random.choice(container[:5], train_images))
valid_files = list(np.random.choice(container[5:], val_images))
tfms = get_transforms(do_flip=True,
flip_vert=True,
max_rotate=45,
max_lighting=0.15,
max_zoom=2
#max_warp=0.0
)
valid = SlideSegmentationItemList(valid_files)
train = SlideSegmentationItemList(train_files)
path = Path('.')
item_list = ItemLists(path, train, valid)
item_list = item_list.label_from_func(get_y_func, classes=['Bg', 'Bone', 'Cartilage', 'PNST', 'Mast Cell Tumor', 'Inflamm/Necrosis', 'Melanoma', 'Plasmacytoma', 'SCC', 'Trichoblastoma', 'Dermis', 'Epidermis', 'Subcutis'], label_cls=SlideSegmentationLabelList) #classes=['Bg', 'Vesel'],
data = item_list.transform(tfms, size=patch_size, tfm_y=True)
data = data.databunch(bs=8) #, num_workers=0
data = data.normalize()
In [ ]:
data.show_batch(6, figsize=(16,16))
Train Model¶
In [ ]:
from sklearn.metrics import jaccard_score as jsc
def iou(outputs: torch.Tensor, labels: torch.Tensor):
outputs = outputs.argmax(dim=1)
labels = labels.squeeze(1)
return tensor(np.mean(jsc(to_np(outputs.view(-1)),to_np(labels.view(-1)), average=None)))
In [ ]:
learn = unet_learner(data, models.resnet18, metrics=[iou], callback_fns=[ShowGraph])
Downloading: "https://download.pytorch.org/models/resnet18-5c106cde.pth" to /root/.cache/torch/hub/checkpoints/resnet18-5c106cde.pth
HBox(children=(FloatProgress(value=0.0, max=46827520.0), HTML(value='')))
In [ ]:
learn.lr_find()
learn.recorder.plot(suggestion=True)
0.00% [0/1 00:00<00:00]
| epoch | train_loss | valid_loss | iou | time |
|---|
59.20% [74/125 00:30<00:21 3.2484]
LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.
Min numerical gradient: 6.31E-05
Min loss divided by 10: 3.31E-03
In [ ]:
learn.fit_one_cycle(1, 1e-4)
| epoch | train_loss | valid_loss | iou | time |
|---|---|---|---|---|
| 0 | 1.108624 | 1.499691 | 0.174103 | 01:08 |
In [ ]:
show_results(learn)
In [ ]:
learn.unfreeze()
learn.fit_one_cycle(10, 1e-4)
| epoch | train_loss | valid_loss | iou | time |
|---|---|---|---|---|
| 0 | 0.895782 | 1.356800 | 0.223802 | 01:10 |
| 1 | 0.858642 | 1.276376 | 0.306351 | 01:09 |
| 2 | 0.840325 | 1.218906 | 0.358768 | 01:09 |
| 3 | 0.789843 | 1.306319 | 0.351105 | 01:09 |
| 4 | 0.732746 | 1.117152 | 0.363201 | 01:09 |
| 5 | 0.646755 | 1.292093 | 0.360563 | 01:08 |
| 6 | 0.625231 | 1.224571 | 0.375784 | 01:08 |
| 7 | 0.563864 | 1.218874 | 0.388219 | 01:09 |
| 8 | 0.552703 | 1.184769 | 0.384153 | 01:09 |
| 9 | 0.550685 | 1.215523 | 0.384732 | 01:09 |
Lets take a look a some results :)¶
In [ ]:
show_results(learn)
In [ ]:
learn.fit_one_cycle(10, 1e-4)
| epoch | train_loss | valid_loss | iou | time |
|---|---|---|---|---|
| 0 | 0.523122 | 1.325038 | 0.377517 | 01:09 |
| 1 | 0.622068 | 1.317604 | 0.366001 | 01:08 |
| 2 | 0.633262 | 1.204316 | 0.370380 | 01:08 |
| 3 | 0.667126 | 1.452963 | 0.357138 | 01:08 |
| 4 | 0.658524 | 1.203093 | 0.364946 | 01:08 |
| 5 | 0.605150 | 1.332683 | 0.353526 | 01:08 |
| 6 | 0.519914 | 1.294832 | 0.370755 | 01:08 |
| 7 | 0.484243 | 1.315896 | 0.378201 | 01:08 |
| 8 | 0.498934 | 1.262955 | 0.398260 | 01:08 |
| 9 | 0.488722 | 1.139007 | 0.399674 | 01:08 |
In [ ]:
learn.fit_one_cycle(10, 1e-3)
| epoch | train_loss | valid_loss | iou | time |
|---|---|---|---|---|
| 0 | 0.639048 | 2.079635 | 0.330621 | 01:08 |
| 1 | 0.857758 | 1.394441 | 0.314440 | 01:08 |
| 2 | 1.008222 | 1.342688 | 0.312155 | 01:08 |
| 3 | 0.969619 | 1.527736 | 0.329167 | 01:07 |
| 4 | 0.897475 | 1.523981 | 0.303677 | 01:07 |
| 5 | 0.746580 | 1.271280 | 0.309353 | 01:07 |
| 6 | 0.714588 | 1.256037 | 0.356753 | 01:07 |
| 7 | 0.695050 | 1.320274 | 0.322581 | 01:07 |
| 8 | 0.638358 | 1.281434 | 0.360952 | 01:07 |
| 9 | 0.606564 | 1.262605 | 0.358417 | 01:07 |
