This document aims to provide an overview of the different data sets available in the Open Data Cube Sandbox. It will provide details on which products are available, what regions are indexed for each product, and the time series available within that extent.
For information on how to get started with the Datacube, please see the read_me_first.ipynb available here.
from IPython.core.display import display, HTML
import pandas as pd
import datacube
dc = datacube.Datacube()
dc.list_products()
name | description | creation_time | lat | platform | lon | format | time | instrument | product_type | label | crs | resolution | tile_size | spatial_dimensions | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
id | |||||||||||||||
15 | alos2_palsar_AMA_ingest | ALOS-2 PALSAR mosaic tiles generated for use i... | None | None | ALOS_2 | None | GeoTIFF | None | PALSAR | gamma0 | None | NaN | NaN | NaN | NaN |
6 | ls5_level1_usgs | Landsat 5 USGS Level 1 Collection-1 OLI-TIRS | None | None | LANDSAT_5 | None | GeoTiff | None | TM | L1TP | None | NaN | NaN | NaN | NaN |
19 | ls5_usgs_sr_scene | Landsat 5 USGS Collection 1 Level2 Surface Ref... | None | None | LANDSAT_5 | None | GeoTiff | None | TM | LEVEL2_USGS | None | NaN | NaN | NaN | NaN |
17 | ls7_collection1_AMA_ingest | Landsat 7 USGS Collection 1 Higher Level SR sc... | None | None | LANDSAT_7 | None | GeoTiff | None | ETM | LEDAPS | None | NaN | NaN | NaN | NaN |
5 | ls7_level1_usgs | Landsat 7 USGS Level 1 Collection-1 OLI-TIRS | None | None | LANDSAT_7 | None | GeoTiff | None | ETM | L1TP | None | NaN | NaN | NaN | NaN |
18 | ls7_usgs_sr_scene | Landsat 7 USGS Collection 1 Level2 Surface Ref... | None | None | LANDSAT_7 | None | GeoTiff | None | ETM | LEVEL2_USGS | None | NaN | NaN | NaN | NaN |
16 | ls8_collection1_AMA_ingest | Landsat 8 USGS Collection 1 Higher Level SR sc... | None | None | LANDSAT_8 | None | GeoTIFF | None | OLI_TIRS | LaSRC | None | NaN | NaN | NaN | NaN |
7 | ls8_l1_pc_usgs | Landsat 8 USGS Level 1 Pre-Collection OLI-TIRS | None | None | LANDSAT_8 | None | GeoTiff | None | OLI_TIRS | L1T | None | NaN | NaN | NaN | NaN |
4 | ls8_level1_usgs | Landsat 8 USGS Level 1 Collection-1 OLI-TIRS | None | None | LANDSAT_8 | None | GeoTiff | None | OLI_TIRS | L1TP | None | NaN | NaN | NaN | NaN |
21 | ls8_usgs_sr_scene | Landsat 8 USGS Collection 1 Higher Level SR sc... | None | None | LANDSAT_8 | None | GeoTiff | None | OLI_TIRS | LEVEL2_USGS | None | NaN | NaN | NaN | NaN |
13 | s1_gamma0_scene | Sentinel-1A/B SAR Gamma0 scenes, processed to ... | None | None | SENTINEL_1 | None | GeoTIFF | None | SAR | gamma0 | None | NaN | NaN | NaN | NaN |
Caqueta
Time: 2015-01-01 and 2016-01-01
latitude: ( 0, 3.2)
longitude: (-76.26666668, -73.6)
Vietnam
Time: 2015-01-01 and 2016-01-01
latitude: ( 7.46666667, 12.8)
longitude: (103.46666668, 110.4)
No Data Indexed
Time: 1984-05-26 to 2011-11-13
y: (-587700.0, 2187300.0)
x: ( 68100.0, 932400.0)
Caqueta
Time: 1999-08-21 to 2018-03-25
latitude: ( 0.000134747292617865, 1.077843593651382)
longitude: (-74.91935994831539, -73.30266193148462)
resolution: ( -0.000269494585236, 0.000269494585236)
Lake Baringo, Kenya
Time: 2005-01-08 to 2016-12-24
latitude: ( 0.4997747685, 0.7495947795)
longitude: (35.9742163305, 36.473586859499996)
resolution: (-0.000269493, 0.000269493)
No Data Indexed
Time: 1999-07-08 to 2018-11-02
y: (-586200.0, 2185200.0)
x: ( 70800.0, 935100.0)
Caqueta
Time: 2013-04-13 to 2018-03-26
latitude: ( 0.000134747292617865, 1.077843593651382)
longitude: (-74.91935994831539, -73.30266193148462)
resolution: ( -0.000269494585236, 0.000269494585236)
Vietnam
Time: 2014-01-14 to 2016-12-21
latitude: ( 10.513927001104687, 12.611133863411238)
longitude: (106.79005909290998, 108.91906631627438)
resolution: ( -0.000269494585236, 0.000269494585236)
No Data Indexed
Time: 2013-03-21 to 2018-11-16
y: (-596700.0, 2193300.0)
x: ( 82500.0, 930300.0)
Time: 2015-05-12 to 2018-07-20
Caqueta
latitude: ( 1.00018083, 2.)
longitude: (-75., -74.)
Samoa
latitude: ( -13.99981917, -13.)
longitude: (-171., -173.)
The following code generates an overview of the approximte extents of the product selected on line 1, shown in red. The yellow base image shows the extents of the global ls8_level1_usgs product.
product = 'ls8_collection1_AMA_ingest'
import matplotlib.pyplot as plt
from pyproj import Proj, transform
import datacube
dc = datacube.Datacube()
outProj = Proj(init='epsg:4326')
x_bounds = []
y_bounds = []
unique_bounds=set()
bounds=[]
times=[]
tiles = dc.find_datasets(product=product)
if max(tiles[0].bounds) > 180: XY = True
else: XY = False
for tile in tiles:
if XY:
inProj = Proj(init=tile.crs)
l_lon,t_lat = transform(inProj,outProj,tile.bounds.left,tile.bounds.top)
r_lon,b_lat = transform(inProj,outProj,tile.bounds.right,tile.bounds.bottom)
x_bounds = [l_lon, r_lon, r_lon, l_lon]
y_bounds = [t_lat, t_lat, b_lat, b_lat]
else:
x_bounds = [tile.bounds.left, tile.bounds.right, tile.bounds.right, tile.bounds.left]
y_bounds = [tile.bounds.top, tile.bounds.top, tile.bounds.bottom, tile.bounds.bottom]
times.append(tile.time)
bounds_key = "".join(str(x) for x in x_bounds) + "".join(str(y) for y in y_bounds)
if bounds_key not in unique_bounds:
bounds.append([x_bounds, y_bounds, tile.time])
%matplotlib inline
fig, ax = plt.subplots(1, figsize = (18,9))
for i in bounds:
ax.fill(i[0], i[1], c='red')
ax.axis('equal')
plt.title(product)
ax.set_xlabel('Longitude')
ax.set_ylabel('Latitude')
basemap = plt.imread('../images/indexed_areas.png')
ax.imshow(basemap, extent=[-180,180,-90,90])
ax.set_xlim(-180,180)
ax.set_ylim(-90, 90)
plt.show()
print('Time:', min(times[-1]).strftime('%Y-%m-%d'), 'to', max(times[0]).strftime('%Y-%m-%d'))
Time: 2013-04-13 to 2018-03-26
import sys
print('Python Version', sys.version)
Python Version 3.6.7 (default, Oct 22 2018, 11:32:17) [GCC 8.2.0]
!pip3 show datacube xarray numpy pandas gdal matplotlib
Name: datacube Version: 1.6.1+179.gc4ed2929.dirty Summary: An analysis environment for satellite and other earth observation data Home-page: https://github.com/opendatacube/datacube-core Author: Open Data Cube Author-email: None License: Apache License 2.0 Location: /usr/local/lib/python3.6/dist-packages Requires: affine, cachetools, click, cloudpickle, dask, gdal, jsonschema, netcdf4, numpy, psycopg2, pypeg2, python-dateutil, pyyaml, rasterio, singledispatch, sqlalchemy, toolz, xarray --- Name: xarray Version: 0.10.7 Summary: N-D labeled arrays and datasets in Python Home-page: https://github.com/pydata/xarray Author: xarray Developers Author-email: xarray@googlegroups.com License: Apache Location: /usr/local/lib/python3.6/dist-packages Requires: pandas, numpy --- Name: numpy Version: 1.15.4 Summary: NumPy: array processing for numbers, strings, records, and objects. Home-page: http://www.numpy.org Author: Travis E. Oliphant et al. Author-email: None License: BSD Location: /home/jovyan/.local/lib/python3.6/site-packages Requires: --- Name: pandas Version: 0.23.4 Summary: Powerful data structures for data analysis, time series, and statistics Home-page: http://pandas.pydata.org Author: None Author-email: None License: BSD Location: /home/jovyan/.local/lib/python3.6/site-packages Requires: python-dateutil, pytz, numpy --- Name: GDAL Version: 2.3.2 Summary: GDAL: Geospatial Data Abstraction Library Home-page: http://www.gdal.org Author: Howard Butler Author-email: hobu.inc@gmail.com License: MIT Location: /usr/lib/python3/dist-packages Requires: --- Name: matplotlib Version: 3.0.2 Summary: Python plotting package Home-page: http://matplotlib.org Author: John D. Hunter, Michael Droettboom Author-email: matplotlib-users@python.org License: BSD Location: /home/jovyan/.local/lib/python3.6/site-packages Requires: pyparsing, cycler, python-dateutil, numpy, kiwisolver
#Generates text extents of data
product = 's1_gamma0_scene'
import datacube
dc = datacube.Datacube()
import numpy as np
left = np.array([])
bottom = np.array([])
right = np.array([])
top = np.array([])
times = np.array([])
tiles = dc.find_datasets(product=product)
for tile in tiles: #Each tile (unique time for each)
left = np.append(left, tile.bounds[0])
right = np.append(right, tile.bounds[2])
top = np.append(top, tile.bounds[3])
bottom = np.append(bottom, tile.bounds[1])
times = np.append(times, tile.time)
#corners = np.append(corners, [])
print('Max Extent')
print('``Time:', min(times).strftime('%Y-%m-%d'), 'to', max(times).strftime('%Y-%m-%d')+'`` ')
print('``latitude: ('+str(min(np.unique(bottom)))+', '+str(max(np.unique(top)))+')`` ')
print('``longitude: ('+str(min(np.unique(left)))+', '+str(max(np.unique(right)))+')`` ')
print('Discrete Extents')
print('``Time:', min(times).strftime('%Y-%m-%d'), 'to', max(times).strftime('%Y-%m-%d')+'`` ')
print('``latitude: ('+str(np.unique(bottom))+', '+str(np.unique(top))+')`` ')
print('``longitude: ('+str(np.unique(left))+', '+str(np.unique(right))+')`` ')
Max Extent ``Time: 2015-05-12 to 2018-07-20`` ``latitude: (-13.9998191681736, 2.0)`` ``longitude: (-173.0, -74.0)`` Discrete Extents ``Time: 2015-05-12 to 2018-07-20`` ``latitude: ([-13.99981917 1.00018083], [-13. 2.])`` ``longitude: ([-173. -172. -75.], [-172. -171. -74.])``
pd.options.display.max_rows = 200
dc.list_measurements()
aliases | dtype | flags_definition | name | nodata | units | ||
---|---|---|---|---|---|---|---|
product | measurement | ||||||
alos2_palsar_AMA_ingest | hh | NaN | int16 | NaN | hh | 0 | DN |
hv | NaN | int16 | NaN | hv | 0 | DN | |
date | NaN | int16 | NaN | date | 0 | 1 | |
incidence_angle | NaN | uint8 | NaN | incidence_angle | 0 | 1 | |
mask | NaN | uint8 | {'cloud_confidence': {'bits': [0, 1, 2, 3, 4, ... | mask | 0 | 1 | |
ls5_level1_usgs | blue | [band_1, blue] | int16 | NaN | blue | -9999 | 1 |
green | [band_2, green] | int16 | NaN | green | -9999 | 1 | |
red | [band_3, red] | int16 | NaN | red | -9999 | 1 | |
nir | [band_4, nir] | int16 | NaN | nir | -9999 | 1 | |
swir1 | [band_5, swir1] | int16 | NaN | swir1 | -9999 | 1 | |
swir2 | [band_7, swir2] | int16 | NaN | swir2 | -9999 | 1 | |
quality | [QUALITY, quality] | int16 | {'cloud': {'bits': [4], 'values': {'0': False,... | quality | 0 | 1 | |
ls5_usgs_sr_scene | blue | [band_1, sr_band1] | int16 | NaN | blue | -9999 | reflectance |
green | [band_2, sr_band2] | int16 | NaN | green | -9999 | reflectance | |
red | [band_3, sr_band3] | int16 | NaN | red | -9999 | reflectance | |
nir | [band_4, sr_band4] | int16 | NaN | nir | -9999 | reflectance | |
swir1 | [band_5, sr_band5] | int16 | NaN | swir1 | -9999 | reflectance | |
swir2 | [band_7, sr_band7] | int16 | NaN | swir2 | -9999 | reflectance | |
sr_atmos_opacity | [atmos_op] | uint8 | NaN | sr_atmos_opacity | 0 | 1 | |
pixel_qa | [pixel_qa] | uint16 | {'snow': {'bits': 4, 'values': {'0': 'no_snow'... | pixel_qa | 1 | bit_index | |
radsat_qa | [radsat_qa] | uint8 | {'radsat_qa': {'bits': [0, 1, 2, 3, 4, 5, 6, 7... | radsat_qa | 1 | bit_index | |
sr_cloud_qa | [pixel_qa] | uint16 | {'snow': {'bits': 4, 'values': {'0': 'no_snow'... | sr_cloud_qa | 1 | bit_index | |
ls7_collection1_AMA_ingest | blue | NaN | int16 | NaN | blue | -9999 | reflectance |
green | NaN | int16 | NaN | green | -9999 | reflectance | |
red | NaN | int16 | NaN | red | -9999 | reflectance | |
nir | NaN | int16 | NaN | nir | -9999 | reflectance | |
swir1 | NaN | int16 | NaN | swir1 | -9999 | reflectance | |
swir2 | NaN | int16 | NaN | swir2 | -9999 | reflectance | |
atmos_opacity | NaN | uint8 | NaN | atmos_opacity | 0 | unitless | |
pixel_qa | NaN | int32 | {'snow': {'bits': 4, 'values': {'0': 'no_snow'... | pixel_qa | 1 | bit_index | |
radsat_qa | NaN | uint8 | {'radsat_qa': {'bits': [0, 1, 2, 3, 4, 5, 6, 7... | radsat_qa | 1 | bit_index | |
cloud_qa | NaN | uint8 | {'cloud_qa': {'bits': [0, 1, 2, 3, 4, 5, 6, 7]... | cloud_qa | 0 | bit_index | |
ls7_level1_usgs | blue | [band_1, blue] | int16 | NaN | blue | -9999 | 1 |
green | [band_2, green] | int16 | NaN | green | -9999 | 1 | |
red | [band_3, red] | int16 | NaN | red | -9999 | 1 | |
nir | [band_4, nir] | int16 | NaN | nir | -9999 | 1 | |
swir1 | [band_5, swir1] | int16 | NaN | swir1 | -9999 | 1 | |
swir2 | [band_7, swir2] | int16 | NaN | swir2 | -9999 | 1 | |
quality | [QUALITY, quality] | int16 | {'cloud': {'bits': [4], 'values': {'0': False,... | quality | 0 | 1 | |
ls7_usgs_sr_scene | blue | [band_1, sr_band1] | int16 | NaN | blue | -9999 | reflectance |
green | [band_2, sr_band2] | int16 | NaN | green | -9999 | reflectance | |
red | [band_3, sr_band3] | int16 | NaN | red | -9999 | reflectance | |
nir | [band_4, sr_band4] | int16 | NaN | nir | -9999 | reflectance | |
swir1 | [band_5, sr_band5] | int16 | NaN | swir1 | -9999 | reflectance | |
swir2 | [band_7, sr_band7] | int16 | NaN | swir2 | -9999 | reflectance | |
sr_atmos_opacity | [atmos_op] | uint8 | NaN | sr_atmos_opacity | 0 | 1 | |
pixel_qa | [pixel_qa] | uint16 | {'snow': {'bits': 4, 'values': {'0': 'no_snow'... | pixel_qa | 1 | bit_index | |
radsat_qa | [radsat_qa] | uint8 | {'radsat_qa': {'bits': [0, 1, 2, 3, 4, 5, 6, 7... | radsat_qa | 1 | bit_index | |
sr_cloud_qa | [cloud_qa] | uint8 | {'cloud_qa': {'bits': [0, 1, 2, 3, 4, 5, 6, 7]... | sr_cloud_qa | 0 | bit_index | |
ls8_collection1_AMA_ingest | coastal_aerosol | NaN | int16 | NaN | coastal_aerosol | -9999 | reflectance |
blue | NaN | int16 | NaN | blue | -9999 | reflectance | |
green | NaN | int16 | NaN | green | -9999 | reflectance | |
red | NaN | int16 | NaN | red | -9999 | reflectance | |
nir | NaN | int16 | NaN | nir | -9999 | reflectance | |
swir1 | NaN | int16 | NaN | swir1 | -9999 | reflectance | |
swir2 | NaN | int16 | NaN | swir2 | -9999 | reflectance | |
pixel_qa | NaN | int32 | {'snow': {'bits': 4, 'values': {'0': 'no_snow'... | pixel_qa | 1 | bit_index | |
aerosol_qa | NaN | int16 | {'aerosol_qa': {'bits': [0, 1, 2, 3, 4, 5, 6, ... | aerosol_qa | 0 | bit_index | |
radsat_qa | NaN | int32 | {'radsat_qa': {'bits': [0, 1, 2, 3, 4, 5, 6, 7... | radsat_qa | 1 | bit_index | |
ls8_l1_pc_usgs | coastal_aerosol | [band_1, coastal_aerosol] | int16 | NaN | coastal_aerosol | -9999 | 1 |
blue | [band_2, blue] | int16 | NaN | blue | -9999 | 1 | |
green | [band_3, green] | int16 | NaN | green | -9999 | 1 | |
red | [band_4, red] | int16 | NaN | red | -9999 | 1 | |
nir | [band_5, nir] | int16 | NaN | nir | -9999 | 1 | |
swir1 | [band_6, swir1] | int16 | NaN | swir1 | -9999 | 1 | |
swir2 | [band_7, swir2] | int16 | NaN | swir2 | -9999 | 1 | |
panchromatic | [band_8, panchromatic] | int16 | NaN | panchromatic | -9999 | 1 | |
cirrus | [band_9, cirrus] | int16 | NaN | cirrus | -9999 | 1 | |
lwir1 | [band_10, lwir1] | int16 | NaN | lwir1 | -9999 | 1 | |
lwir2 | [band_11, lwir2] | int16 | NaN | lwir2 | -9999 | 1 | |
quality | [QUALITY, quality] | int16 | {'water': {'bits': [4, 5], 'values': {'0': Fal... | quality | 0 | 1 | |
ls8_level1_usgs | coastal_aerosol | [band_1, coastal_aerosol] | int16 | NaN | coastal_aerosol | -9999 | 1 |
blue | [band_2, blue] | int16 | NaN | blue | -9999 | 1 | |
green | [band_3, green] | int16 | NaN | green | -9999 | 1 | |
red | [band_4, red] | int16 | NaN | red | -9999 | 1 | |
nir | [band_5, nir] | int16 | NaN | nir | -9999 | 1 | |
swir1 | [band_6, swir1] | int16 | NaN | swir1 | -9999 | 1 | |
swir2 | [band_7, swir2] | int16 | NaN | swir2 | -9999 | 1 | |
panchromatic | [band_8, panchromatic] | int16 | NaN | panchromatic | -9999 | 1 | |
cirrus | [band_9, cirrus] | int16 | NaN | cirrus | -9999 | 1 | |
lwir1 | [band_10, lwir1] | int16 | NaN | lwir1 | -9999 | 1 | |
lwir2 | [band_11, lwir2] | int16 | NaN | lwir2 | -9999 | 1 | |
quality | [QUALITY, quality] | int16 | {'cloud': {'bits': [4], 'values': {'0': False,... | quality | 0 | 1 | |
ls8_usgs_sr_scene | coastal_aerosol | [band_1, sr_band1] | int16 | NaN | coastal_aerosol | -9999 | reflectance |
blue | [band_2, sr_band2] | int16 | NaN | blue | -9999 | reflectance | |
green | [band_3, sr_band3] | int16 | NaN | green | -9999 | reflectance | |
red | [band_4, sr_band4] | int16 | NaN | red | -9999 | reflectance | |
nir | [band_5, sr_band5] | int16 | NaN | nir | -9999 | reflectance | |
swir1 | [band_6, sr_band6] | int16 | NaN | swir1 | -9999 | reflectance | |
swir2 | [band_7, sr_band7] | int16 | NaN | swir2 | -9999 | reflectance | |
pixel_qa | [pixel_qa] | uint16 | {'snow': {'bits': 4, 'values': {'0': 'no_snow'... | pixel_qa | 1 | bit_index | |
sr_aerosol | [sr_aerosol_qa, aerosol_qa, aerosol] | uint8 | {'aerosol_qa': {'bits': [0, 1, 2, 3, 4, 5, 6, ... | sr_aerosol | 0 | bit_index | |
radsat_qa | [radsat_qa] | uint16 | {'radsat_qa': {'bits': [0, 1, 2, 3, 4, 5, 6, 7... | radsat_qa | 1 | bit_index | |
s1_gamma0_scene | vh | NaN | float32 | NaN | vh | 0 | dB |
vv | NaN | float32 | NaN | vv | 0 | dB |