Notebook

Using the Radiant MLHub Models API¶

In addition to training datasets, Radiant MLHub also gives access to machine learning models that generate predictions from EO data. You can learn more about the available models and how to query them in this blog post.

This Jupyter notebook, which you may copy and adapt for any use, shows basic examples of how to use the API to find and use models. Full documentation for the API is available at docs.mlhub.earth.

We'll show you how to set up your authentication, see the list of available models, and retrieve the individual models by ID.

Authentication¶

Access to the Radiant MLHub API requires an API key. To get your API key, go to mlhub.earth and click the "Sign in / Register" button in the top right to log in. If you have not used Radiant MLHub before, you will need to sign up and create a new account; otherwise, just sign in. Once you have signed in, click on your user avatar in the top right and select the "Settings & API keys" from the dropdown menu.

In the API Keys section of this page, you will be able to create new API key(s). Do not share your API key with others as this may pose a security risk.

Next, we will create a MLHUB_API_KEY variable that pystac-client will use later use to add our API key to all requests:

In [1]:

import getpass

MLHUB_API_KEY = getpass.getpass(prompt="MLHub API Key: ")
MLHUB_ROOT_URL = "https://api.radiant.earth/mlhub/v1"

MLHub API Key:  ································································

Finally, we connect to the Radiant MLHub API using our API key:

In [2]:

import itertools as it
import requests
from pprint import pprint
from urllib.parse import urljoin

from pystac_client import Client
from pystac import StacIO
from pystac.extensions.scientific import ScientificExtension

client = Client.open(
    MLHUB_ROOT_URL, parameters={"key": MLHUB_API_KEY}, ignore_conformance=True
)

List models¶

A model in the Radiant MLHub API is a STAC Item that implements the ML Model Extension. This extensions provides high-level metadata about the model, including a description of the architecture and training environment. It also includes links to the datasets on which the model was trained.

We start by creating a requests.Session instance so that we can include the API key in all of our requests.

In [3]:

class MLHubSession(requests.Session):
    def __init__(self, *args, api_key=None, **kwargs):
        super().__init__(*args, **kwargs)
        self.params.update({"key": api_key})

    def request(self, method, url, *args, **kwargs):
        url_prefix = MLHUB_ROOT_URL.rstrip("/") + "/"
        url = urljoin(url_prefix, url)
        return super().request(method, url, *args, **kwargs)


session = MLHubSession(api_key=MLHUB_API_KEY)

Next, we list the available models using the /models endpoint

In [4]:

response = session.get("/models")
models = response.json()

models_limit = 30

print(f"Total Datasets: {len(models)}")
print("-----")
for model in it.islice(models, models_limit):
    model_id = model["id"]
    print(f"{model_id}")
if len(models) > models_limit:
    print("...")

Total Datasets: 1
-----
model-cyclone-wind-estimation-torchgeo-v1

Model metadata¶

We can also use the STAC API Item Search capability to a fetch model metadata by ID. Let's fetch the Tropical Cyclone Model using its ID.

In [5]:

results = client.search(ids=["model-cyclone-wind-estimation-torchgeo-v1"])
cyclone_model = next(results.get_items())
pprint(cyclone_model.to_dict())

{'assets': {'inferencing-checkpoint': {'href': 'https://api.radiant.earth/mlhub/v1/download/gAAAAABh1Lo2NQfk1Km_1oYLIx7_EdvyKXpHQZ0Hq2_Tj4gZppf4GGa2RFI9xXQP5x_aVRxXoYPqThgrZwgrVEtC6E_V9gl0-aVdxPY2rhg82Ahm2ap1OfmD2hT2IjM270TqQ_qYCu5httnFFRNaUKncz6BOKH7spCf8Agdp5-AnBtq-99LvDP2B_M_T5guUIK3h8pg1YzbIWXVTLAzkrFubZO1yRde9Hw==',
                                       'roles': ['ml-model:checkpoint'],
                                       'title': 'Final model checkpoint',
                                       'type': 'application/octet-stream'},
            'inferencing-compose': {'href': 'https://api.radiant.earth/mlhub/v1/download/gAAAAABh1Lo2SptgZTV6PvAyDE3FNfhyme0nsLIXt-CcGOTcUWfHOUp37prpshZh2U7bm0qR-JsnwkRCfjq2RfFilWX2uscNafq2c6fmy25CZYEdOghu-l0eQPeguKvRlzcdQ8eCZP4Y9PAr2httfkCaT8Be16YyaFtI2OErhc0mAWZ38WPRBenrmvuJXGQTrcMwDoR3ncigfX_Lp1iCONWYgcAaoCpOR1I5L49IM3q7MzR0wKk1XQRYMPZaiIYMWXnM5ziOtnUz',
                                    'roles': ['ml-model:inference-runtime'],
                                    'title': 'Model inferencing runtime',
                                    'type': 'text/x-yaml; '
                                            'application=compose'}},
 'bbox': [-179.999, -89.999, 179.999, 89.999],
 'collection': 'model-cyclone-wind-estimation-torchgeo',
 'geometry': {'coordinates': [[[-179.999, -89.999],
                               [179.999, -89.999],
                               [179.999, 89.999],
                               [-179.999, 89.999],
                               [-179.999, -89.999]]],
              'type': 'Polygon'},
 'id': 'model-cyclone-wind-estimation-torchgeo-v1',
 'links': [{'href': 'http://api.radiant.earth/mlhub/v1/collections/model-cyclone-wind-estimation-torchgeo',
            'rel': 'collection',
            'type': 'application/json'},
           {'href': 'http://api.radiant.earth/mlhub/v1/collections/model-cyclone-wind-estimation-torchgeo',
            'rel': 'parent',
            'type': 'application/json'},
           {'href': 'https://api.radiant.earth/mlhub/v1',
            'rel': <RelType.ROOT: 'root'>,
            'title': 'Radiant MLHub API',
            'type': <MediaType.JSON: 'application/json'>},
           {'href': 'http://api.radiant.earth/mlhub/v1/collections/model-cyclone-wind-estimation-torchgeo/items/model-cyclone-wind-estimation-torchgeo-v1',
            'rel': 'self',
            'type': 'application/geo+json'},
           {'href': 'http://api.radiant.earth/mlhub/v1/collections/model-cyclone-wind-estimation-torchgeo/items/model-cyclone-wind-estimation-torchgeo-v1/tiles',
            'rel': 'alternate',
            'title': 'tiles',
            'type': 'application/json'}],
 'properties': {'datetime': None,
                'end_datetime': '2019-12-12T23:59:59Z',
                'license': 'MIT',
                'ml-model:architecture': 'resnet18',
                'ml-model:learning_approach': 'supervised',
                'ml-model:prediction_type': 'regression',
                'ml-model:type': 'ml-model',
                'providers': [{'email': 'caleb.robinson@microsoft.com',
                               'name': 'Microsoft AI for Good Research Lab '
                                       '(Caleb Robinson)',
                               'roles': ['producer'],
                               'url': 'https://www.microsoft.com/en-us/ai/ai-for-good'}],
                'sci:citation': 'Caleb Robinson. (2021). Tropical Cyclone Wind '
                                'Estimation model (2.0). Zenodo. '
                                'https://doi.org/10.5281/zenodo.5773331.',
                'sci:doi': '10.5281/zenodo.5773331',
                'sci:publications': [{'citation': 'Stewart, A., Robinson, C. '
                                                  'and Corley, I., 2021. '
                                                  'TorchGeo: deep learning '
                                                  'with geospatial data. arXiv '
                                                  'preprint arXiv:2111.08872, '
                                                  '[online] (11). Available '
                                                  'at: '
                                                  '<https://arxiv.org/abs/2111.08872> '
                                                  '[Date Accessed].'}],
                'start_datetime': '2000-01-01T00:00:00Z'},
 'stac_extensions': ['https://stac-extensions.github.io/ml-model/v1.0.0/schema.json',
                     'https://stac-extensions.github.io/scientific/v1.0.0/schema.json'],
 'stac_version': '1.0.0',
 'type': 'Feature'}

There is a lot of metadata here, but let's walk through some of the key fields and what they mean.

Spatiotemporal Metadata¶

The geometry and bbox fields represent geographic areas over which the model can be used. Similarly, the start_datetime and end_datetime properties represent the temporal range of data that be used by the model. The special value 9999-12-31T23:59:59Z for the end_datetime property means that any date after the start_datetime will work.

In [6]:

print(f"Bounding Box: {cyclone_model.bbox}")
print(f"Geometry: {cyclone_model.geometry}")

start_datetime = cyclone_model.to_dict()["properties"]["start_datetime"]
end_datetime = cyclone_model.to_dict()["properties"]["end_datetime"]
end_datetime = end_datetime if end_datetime != "9999-12-31T23:59:59Z" else "..."

print(f"Datetime Range: {start_datetime}/{end_datetime}")

Bounding Box: [-179.999, -89.999, 179.999, 89.999]
Geometry: {'type': 'Polygon', 'coordinates': [[[-179.999, -89.999], [179.999, -89.999], [179.999, 89.999], [-179.999, 89.999], [-179.999, -89.999]]]}
Datetime Range: 2000-01-01T00:00:00Z/2019-12-12T23:59:59Z

Citation Information¶

Citation and publication metadata related to the model will be described using the STAC Scientific Citation Extension. Let's first check if this Item implements the Scientific Citation Extension, and then print some of the citation metadata if it does.

In [7]:

implements_sci_ext = ScientificExtension.has_extension(cyclone_model)
print(f"Has citation information: {implements_sci_ext}\n")

if implements_sci_ext:
    sci_ext = ScientificExtension.ext(cyclone_model)
    if sci_ext.citation is not None:
        print(f"Citation: {sci_ext.citation}")
    if sci_ext.doi is not None:
        print(f"DOI: {sci_ext.doi}")
    if sci_ext.publications is not None:
        print("\nPublications\n------------")
        for publication in sci_ext.publications:
            if publication.citation is not None:
                print(f"- Citation: {publication.citation}")

Has citation information: True

Citation: Caleb Robinson. (2021). Tropical Cyclone Wind Estimation model (2.0). Zenodo. https://doi.org/10.5281/zenodo.5773331.
DOI: 10.5281/zenodo.5773331

Publications
------------
- Citation: Stewart, A., Robinson, C. and Corley, I., 2021. TorchGeo: deep learning with geospatial data. arXiv preprint arXiv:2111.08872, [online] (11). Available at: <https://arxiv.org/abs/2111.08872> [Date Accessed].

Model Metadata¶

Metadata related to the ML model itself will be described using the STAC ML Model Extension. This extension includes properties describing model architecture and prediction type, and also defines type STAC Link and Asset types that are specific to ML models. Let's take a closer look at what each of these properties represents.

Note: PySTAC does not currently have built-in support for the ML Model Extension, so we will access these properties using the items.properties dictionary directly.

In [8]:

architecture = cyclone_model.properties["ml-model:architecture"]
learning_approach = cyclone_model.properties["ml-model:learning_approach"]
prediction_type = cyclone_model.properties["ml-model:prediction_type"]

print(f"Architecture: {architecture}")
print(f"Learning Approach: {learning_approach}")
print(f"Prediction Type: {prediction_type}")

Architecture: resnet18
Learning Approach: supervised
Prediction Type: regression

We can see that this is based on a residual neural network (ResNet18), that it generates a regression output, and that it was trained using supervised learning.

Model Assets¶

The ML Model Extension describes some STAC Asset types that are specific to ML models. In particular, the Tropical Cyclone Model provides Assets for a model checkpoint and a Compose file, either of which may be used to run the model to generate inferences. These assets have roles of ml-model:checkpoint and ml-model:inference-runtime, respectively. Let's take a look at these Assets and then download the Compose file to see what that looks like.

In [9]:

checkpoint_asset = next(
    asset
    for asset_key, asset in cyclone_model.assets.items()
    if "ml-model:checkpoint" in asset.roles
)

print("Checkpoint Asset\n----------------")
pprint(checkpoint_asset.to_dict())

inference_runtime_asset = next(
    asset
    for asset_key, asset in cyclone_model.assets.items()
    if "ml-model:inference-runtime" in asset.roles
)

print()

print("Inference Runtime Asset\n----------------")
pprint(inference_runtime_asset.to_dict())

Checkpoint Asset
----------------
{'href': 'https://api.radiant.earth/mlhub/v1/download/gAAAAABh1Lo2NQfk1Km_1oYLIx7_EdvyKXpHQZ0Hq2_Tj4gZppf4GGa2RFI9xXQP5x_aVRxXoYPqThgrZwgrVEtC6E_V9gl0-aVdxPY2rhg82Ahm2ap1OfmD2hT2IjM270TqQ_qYCu5httnFFRNaUKncz6BOKH7spCf8Agdp5-AnBtq-99LvDP2B_M_T5guUIK3h8pg1YzbIWXVTLAzkrFubZO1yRde9Hw==',
 'roles': ['ml-model:checkpoint'],
 'title': 'Final model checkpoint',
 'type': 'application/octet-stream'}

Inference Runtime Asset
----------------
{'href': 'https://api.radiant.earth/mlhub/v1/download/gAAAAABh1Lo2SptgZTV6PvAyDE3FNfhyme0nsLIXt-CcGOTcUWfHOUp37prpshZh2U7bm0qR-JsnwkRCfjq2RfFilWX2uscNafq2c6fmy25CZYEdOghu-l0eQPeguKvRlzcdQ8eCZP4Y9PAr2httfkCaT8Be16YyaFtI2OErhc0mAWZ38WPRBenrmvuJXGQTrcMwDoR3ncigfX_Lp1iCONWYgcAaoCpOR1I5L49IM3q7MzR0wKk1XQRYMPZaiIYMWXnM5ziOtnUz',
 'roles': ['ml-model:inference-runtime'],
 'title': 'Model inferencing runtime',
 'type': 'text/x-yaml; application=compose'}

Next, we'll read the Compose file and examine its contents.

In [10]:

stac_io = StacIO.default()
compose_file_contents = stac_io.read_text(inference_runtime_asset.href)
print(compose_file_contents)

services:
  inference:
    image: radiantearth/cyclone-model-torchgeo:1
    volumes:
      - "${INPUT_DATA}:/var/data/input"
      - "${OUTPUT_DATA}:/var/data/output"

You can find more details about running models using Compose files in the "Inference/Training Runtimes" section of the ML Model Extension documentation.

Next Steps¶

This tutorial was a quick introduction to working with the Radiant MLHub Models API in a notebook. For more, see: