Notebook

DotNetStac

.Net library for working with Spatio Temporal Asset Catalogs (STAC)

DotNetStac helps you to work with STAC (catalog, collection, item)¶

In a nutshell, the library allows import/export of STAC JSON documents (Serialization/Deserialization using Newtonsoft.JSON) to typed object with properties represented in enhanced objects such as geometries, time stamp/period/span, numerical values and many more via STAC extension plugins engine.

Getting Started¶

In this notebook, we are going to review the library's features through a series of code examples.

Important Note about .Net programming and C# language usage in this library¶

We chose to design and implement the objects of this library as plain as possible (e.g. POCO) in order to keep the structured and typed nature of the C# language. So, we will deal with normal classes, without more attributes describing infrastructure concerns or other responsibilities that your domain objects shouldn't have.

1 Install DotNetStac¶

First, we either install lastest version of DotNetStac in the notebook or we use the locally built binaries

In [ ]:

// Use built lib
#i "/home/emathot/Workspace/Terradue/Components/sugar/DotNetStac/src/DotNetStac/bin/Debug/netstandard2.0/publish/"
#r "/home/emathot/Workspace/Terradue/Components/sugar/DotNetStac/src/DotNetStac/bin/Debug/netstandard2.0/publish/DotNetStac.dll"

2 Deserialization of STAC documents¶

The (de)serialiation methods are wrapped in methods in class StacConvert that is the main entry point from/to JSON/.Net.

Let's start reading a STAC catalog online. Please note that DotNetStac does not provide with data access middleware. You can integrate own data access or you can test the Stars SDK that provides with integrated functions to manipulate STAC objects and their storage.

The following code is a very simple function loading a catalog and printing it's id, description and stac_version.

In [ ]:

using Stac;
using Stac.Schemas;
using System;
using System.Net;
using Newtonsoft.Json.Schema;

var webc = new WebClient();
Uri catalogUri = new Uri("https://cbers-stac-1-0-rc.s3.amazonaws.com/catalog.json");
StacValidator stacValidator = new StacValidator(new JSchemaUrlResolver());

// StacConvert.Deserialize is the helper to start loading any STAC document
var json = webc.DownloadString(catalogUri);
bool valid = stacValidator.ValidateJson(json);
StacCatalog catalog = StacConvert.Deserialize<StacCatalog>(json);

Console.Out.WriteLine(catalog.Id + ": " + catalog.Description + (valid ? " [VALID]" : "[INVALID]"));
Console.Out.WriteLine(catalog.StacVersion);

Using the previously loaded catalog, the following code executes a recursive function navigating from a root catalog through all it's tree structure recursing in the child STAC links and listing the item links and their assets.

Please note the following:

GetChildrenLinks and GetItemLinks are the recommanded ways to get the links for navigating through the tree.
The previous functions as the rest of the library does not alter the Uris. It is then up to the developer to resolve the relative ones. As in the code, Uri class provides with all the necessary methods to easily join a base Url with a relative one.
The StacConvert.Deserialize<> methods allows to specify the interfaces to ease the deserialization when the STAC type is unknown: IStacObject and IStacCatalog.

In [ ]:

using System.Linq;
using Stac.Extensions.ItemCollections;

public static void ListChildrensItemsAndAssets(IStacParent catalog, Uri baseUri, WebClient webc, StacValidator stacValidator, string prefix = "", int limit = 2)
{
    // Get children and items (sub catalogs, collections and items)
    foreach (var childLink in catalog.GetChildrenLinks().Concat(catalog.GetItemLinks()).Take(limit))
    {
        // IMPORTANT: Relative Uri resolution
        Uri childUri = childLink.Uri;
        if (!childUri.IsAbsoluteUri)
            childUri = new Uri(baseUri, childUri.ToString());
        var childjson = webc.DownloadString(childUri);
        // STAC schema validation
        bool valid = stacValidator.ValidateJson(childjson);
        // STAC object loading (using the IStacObject interface)
        IStacObject child = null;
        try {
            child = StacConvert.Deserialize<IStacObject>(childjson);
        }   
        catch (Exception e) {
            Console.Error.WriteLine(string.Format("Error deserializing STAC object at '{0}' : {1}", childLink.Uri, e.Message));
            Console.Error.WriteLine(e.InnerException.StackTrace);
        }

        if (child is StacCatalog || child is StacCollection)
            Console.Out.WriteLine(prefix + child.Id + ": " + child.Title + (valid ? " [VALID]" : " [INVALID]"));

        List<StacItem> items = new List<StacItem>();

        // Item or ItemCollection
        if (child is StacItem)
            items.Add(child as StacItem);
        if (child is ItemCollection)
            items = (child as ItemCollection).Features;
        // List assets if item
        foreach (var item in items)
        {
            // Print the item
            Console.Out.WriteLine(prefix + "  " + item.Id + ": " + item.Title + (valid ? " [VALID]" : " [INVALID]"));
            foreach (var asset in item.Assets.Values) {
                Console.Out.WriteLine(prefix + "    *[" + asset.MediaType + "] " + asset.Uri);
            }
        }

        // Go deeper if catalog or collection
        if (child is StacCatalog || child is StacCollection){
            ListChildrensItemsAndAssets(child as IStacParent, childUri, webc, stacValidator, prefix + "  ", limit);
        }
        
    }
}

// Start the navigation
ListChildrensItemsAndAssets(catalog as IStacParent, catalogUri, webc, stacValidator);

4 Creation and serialization of STAC documents¶

4.1 Collection creation from scratch¶

Let's now create a STAC collection programmatically using the plain Collection object. Here all collection specific fields are set manually.

In [ ]:

using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using Stac;
using Stac.Collection;
using System;
using System.Collections.Generic;


// First the mandatory elements of a Collection
// Spatial et Temporal Extent
StacExtent extent = new StacExtent(
                new StacSpatialExtent(-180, -56, 180, 83),
                new StacTemporalExtent(DateTime.Parse("2015-06-23T00:00:00Z").ToUniversalTime(), null)
        );

// Create the Collection
StacCollection collection = new StacCollection("COPERNICUS/S2",
                                            "Sentinel-2 is a wide-swath, high-resolution, multi-spectral\nimaging mission supporting Copernicus Land Monitoring studies,\nincluding the monitoring of vegetation, soil and water cover,\nas well as observation of inland waterways and coastal areas.\n\nThe Sentinel-2 data contain 13 UINT16 spectral bands representing\nTOA reflectance scaled by 10000. See the [Sentinel-2 User Handbook](https://sentinel.esa.int/documents/247904/685211/Sentinel-2_User_Handbook)\nfor details. In addition, three QA bands are present where one\n(QA60) is a bitmask band with cloud mask information. For more\ndetails, [see the full explanation of how cloud masks are computed.](https://sentinel.esa.int/web/sentinel/technical-guides/sentinel-2-msi/level-1c/cloud-masks)\n\nEach Sentinel-2 product (zip archive) may contain multiple\ngranules. Each granule becomes a separate Earth Engine asset.\nEE asset ids for Sentinel-2 assets have the following format:\nCOPERNICUS/S2/20151128T002653_20151128T102149_T56MNN. Here the\nfirst numeric part represents the sensing date and time, the\nsecond numeric part represents the product generation date and\ntime, and the final 6-character string is a unique granule identifier\nindicating its UTM grid reference (see [MGRS](https://en.wikipedia.org/wiki/Military_Grid_Reference_System)).\n\nFor more details on Sentinel-2 radiometric resoltuon, [see this page](https://earth.esa.int/web/sentinel/user-guides/sentinel-2-msi/resolutions/radiometric).\n",
                                            extent);

collection.Title = "Sentinel-2 MSI: MultiSpectral Instrument, Level-1C";

collection.Links.Add(StacLink.CreateSelfLink(new Uri("https://storage.cloud.google.com/earthengine-test/catalog/COPERNICUS_S2.json")));
collection.Links.Add(StacLink.CreateParentLink(new Uri("https://storage.cloud.google.com/earthengine-test/catalog/catalog.json")));
collection.Links.Add(StacLink.CreateRootLink(new Uri("https://storage.cloud.google.com/earthengine-test/catalog/catalog.json")));
collection.Links.Add(new StacLink(new Uri("https://scihub.copernicus.eu/twiki/pub/SciHubWebPortal/TermsConditions/Sentinel_Data_Terms_and_Conditions.pdf"), "license", "Legal notice on the use of Copernicus Sentinel Data and Service Information", null));

collection.Keywords.Add("copernicus");
collection.Keywords.Add("esa");
collection.Keywords.Add("eu");
collection.Keywords.Add("msi");
collection.Keywords.Add("radiance");
collection.Keywords.Add("sentinel");

collection.Providers.Add(new StacProvider("European Union/ESA/Copernicus",
                new List<StacProviderRole>() { StacProviderRole.producer, StacProviderRole.licensor })
{
    Uri = new Uri("https://sentinel.esa.int/web/sentinel/user-guides/sentinel-2-msi")
});

collection.Summaries.Add("datetime",
    new StacSummaryRangeObject<DateTime>(
        DateTime.Parse("2015-06-23T00:00:00Z").ToUniversalTime(),
        DateTime.Parse("2019-07-10T13:44:56Z").ToUniversalTime()
    )
);

var platforms = new StacSummaryValueSet<string>();
platforms.Add("sentinel-2a");
platforms.Add("sentinel-2b");

collection.Summaries.Add("platform", platforms);

collection.Summaries.Add("constellation",
    new StacSummaryValueSet<string>(new string[] { "sentinel-2" })
);

collection.Summaries.Add("instruments",
    new StacSummaryValueSet<string>(new string[] { "msi" })
);

collection.Summaries.Add("view:off_nadir",
    new StacSummaryRangeObject<double>(
        0.0,
        100
    )
);

collection.Summaries.Add("view:sun_elevation",
    new StacSummaryRangeObject<double>(
        6.78,
        89.9
    )
);

collection.Summaries.Add("sci:citation",
    new StacSummaryValueSet<string>(new string[] { "Copernicus Sentinel data [Year]" })
);

collection.Summaries.Add("gsd",
    new StacSummaryValueSet<int>(new int[] {
        10,
        30,
        60
    })
);

collection.Summaries.Add("proj:epsg",
    new StacSummaryValueSet<int>(new int[]
        { 32601,32602,32603,32604,32605,32606,32607,32608,32609,32610,32611,32612,32613,32614,32615,32616,32617,32618,32619,32620,32621,32622,32623,32624,32625,32626,32627,32628,32629,32630,32631,32632,32633,32634,32635,32636,32637,32638,32639,32640,32641,32642,32643,32644,32645,32646,32647,32648,32649,32650,32651,32652,32653,32654,32655,32656,32657,32658,32659,32660}
    )
);

collection.Summaries.Add("eo:bands",
    new StacSummaryValueSet<JObject>(new JObject[] {
        new JObject {
            { "name", "B1" },
            { "common_name", "coastal" },
            { "center_wavelength", 4.439 }
        },
        new JObject {
            { "name", "B2"},
            { "common_name", "blue"},
            { "center_wavelength", 4.966}
        },
        new JObject {
            { "name", "B3"},
            { "common_name", "green"},
            { "center_wavelength", 5.6}
        },
        new JObject {
            { "name", "B4"},
            { "common_name", "red"},
            { "center_wavelength", 6.645}
        },
        new JObject {
            { "name", "B5"},
            { "center_wavelength", 7.039}
        },
        new JObject {
            { "name", "B6"},
            { "center_wavelength", 7.402}
        },
        new JObject {
            { "name", "B7"},
            { "center_wavelength", 7.825}
        },
        new JObject {
            { "name", "B8"},
            { "common_name", "nir"},
            { "center_wavelength", 8.351}
        },
        new JObject {
            { "name", "B8A"},
            { "center_wavelength", 8.648}
        },
        new JObject {
            { "name", "B9"},
            { "center_wavelength", 9.45}
        },
        new JObject {
            { "name", "B10"},
            { "center_wavelength", 1.3735}
        },
        new JObject {
            { "name", "B11"},
            { "common_name", "swir16"},
            { "center_wavelength", 1.6137}
        },
        new JObject {
            { "name", "B12"},
            { "common_name", "swir22"},
            { "center_wavelength", 2.2024}
        }
    })
);


// Serialize
JsonSerializerSettings serSettings = new JsonSerializerSettings() { Formatting = Formatting.Indented };
var json = StacConvert.Serialize(collection, serSettings);
// Print JSON!
Console.WriteLine(json)

4.2 Collection generation from a set of Items¶

StacCollection class has static methods allowing the automatic generation of a collection from a set of StacItem. The following code loads the items of the examples folder from STAC repository and generates the corresponding collection with

Spatial and temporal extent from geometry and time merge of the items
Fields summaries with stats objects or value sets of the items' fields values

Please note that the function takes also the eventual uri of the collection in input. If specified, the items Uri are made relative to that uri.

In [ ]:

Uri simpleItemUri = new Uri("https://raw.githubusercontent.com/radiantearth/stac-spec/dev/examples/simple-item.json");
Uri coreItemUri = new Uri("https://raw.githubusercontent.com/radiantearth/stac-spec/dev/examples/core-item.json");
Uri extendedItemUri = new Uri("https://raw.githubusercontent.com/radiantearth/stac-spec/dev/examples/extended-item.json");

StacItem simpleItem = StacConvert.Deserialize<StacItem>(webc.DownloadString(simpleItemUri));
StacItem coreItem = StacConvert.Deserialize<StacItem>(webc.DownloadString(coreItemUri));
StacItem extendedItem = StacConvert.Deserialize<StacItem>(webc.DownloadString(extendedItemUri));

Dictionary<Uri, StacItem> items = new Dictionary<Uri, StacItem>();
items.Add(simpleItemUri, simpleItem);
items.Add(coreItemUri,coreItem);
items.Add(extendedItemUri, extendedItem);

StacCollection stacCollection = StacCollection.Create("simple-collection",
                                                        "A simple collection demonstrating core catalog fields with links to a couple of items",
                                                        items,
                                                        "CC-BY-4.0", null, null);

Console.Out.Write(StacConvert.Serialize(collection, serSettings));

5 STAC Extensions support¶

The library implements STAC extensions as extensions classes for enabling quick fields accessors. A extension class may also implement helpers related to the extensions such as

projection: WKT generation from EPSG identifier
file: file extension fields generation from FileInfo or Stream with Multihash checksum generation
raster: Optical calibration parameters

5.1 Projection extension

In the following example, we create a simple STAC item and we use the projection extension helpers to set the coordinate system fields by:

using the EPSG system reference identifier
using a Proj.Net CoordinateSystem object

In [ ]:

using GeoJSON.Net;
using GeoJSON.Net.Geometry;
using Itenso.TimePeriod;
using ProjNet.CoordinateSystems;
using Stac.Extensions.Projection;

var coordinates = new[]
{
    new List<IPosition>
    {
        new Position(37.488035566,-122.308150179),
        new Position(37.538869539,-122.597502109),
        new Position(37.613537207,-122.576687533),
        new Position(37.562818007,-122.288048600),
        new Position(37.488035566,-122.308150179)
    }
};
var geometry = new Polygon(new LineString[] { new LineString(coordinates[0]) });
StacItem item = new StacItem("CS3-20160503_132130_04", geometry);
item.DateTime = new TimeInterval(DateTime.Parse("2016-05-03T13:21:30.040Z"));
// Set the UTM#33 north coordinate system
item.ProjectionExtension().SetCoordinateSystem(32633);
// or from ProjNet object
item.ProjectionExtension().SetCoordinateSystem(ProjectedCoordinateSystem.WGS84_UTM(33, true));
string json = StacConvert.Serialize(item, serSettings);
stacValidator.ValidateJson(json);
Console.Out.WriteLine(json);

5.1 File extension

In the following example, we create a simple STAC item and we use the file extension helpers to set the file extension fields from a file on the local file system or from a stream.

In [ ]:

using Stac.Extensions.File;
StacAsset stacAsset = StacAsset.CreateDataAsset(item,
                                                new Uri("file:///srid.csv"),
                                                new System.Net.Mime.ContentType("text/csv"),
                                                "System reference Ids");
await stacAsset.FileExtension().SetFileExtensionProperties(new System.IO.FileInfo("SRID.csv"));
item.Assets.Add("srid", stacAsset);
string json = StacConvert.Serialize(item, serSettings);
stacValidator.ValidateJson(json);
Console.Out.WriteLine(json);

Thank you!

www.terradue.com
Emmanuel Mathot
emmanuel.mathot@terradue.com

In [ ]:

DotNetStac

.Net library for working with Spatio Temporal Asset Catalogs (STAC)

DotNetStac helps you to work with STAC (catalog, collection, item)¶

Getting Started¶

Important Note about .Net programming and C# language usage in this library¶

1 Install DotNetStac¶

2 Deserialization of STAC documents¶

3 Navigation from a Catalogue¶

4 Creation and serialization of STAC documents¶

4.1 Collection creation from scratch¶

4.2 Collection generation from a set of Items¶

5 STAC Extensions support¶

Thank you!