The processing of radar data - often referred to as SAR data, which stands for Synthetice Aperture Radar - is a topic of its own and quite different from optical data that uses Red, Green, Blue, NIR and similar bands. This is the first of a series of jupyter notebooks where example SAR workflows will be explained that can be run on UP42. All of the examples are based on a specific type of analysis which is called polarimetry and processes one image at a time. In comparison to polarimetric there also is interferometric analysis which leverages the information encoded in suitable pairs of radar image.
In this notebook we want to have a look at the Sentinel-1 GRD full scene block, followed by SNAP Sentinel-1 Polarimetric Processing. Polarimetric SAR Processing turns a Sentinel-1 GRD image into a multichannel GeoTIFF that is ready for analysis.
import up42
# Authenticate with UP42
up42.authenticate(cfg_file="config.json")
#up42.authenticate(project_id=12345, project_api_key=12345)
2020-05-26 14:19:56,546 - up42.auth - INFO - Got credentials from config file. 2020-05-26 14:19:57,155 - up42.auth - INFO - Authentication with UP42 successful!
Create a new project on UP42 or use an existing one.
S1_SNAP_project = up42.initialize_project()
2020-05-26 14:20:10,960 - up42 - INFO - Working on Project with project_id d3ea9123-2b89-4975-be16-e978fed8329c
Create workflow and check available blocks and data
workflow = S1_SNAP_project.create_workflow(name="S1-GRD_SNAP",
use_existing=True)
print(up42.get_blocks(basic=True))
2020-05-26 14:20:14,765 - up42.project - INFO - Getting existing workflows in project ... 2020-05-26 14:20:15,434 - up42.project - INFO - Got 12 workflows for project d3ea9123-2b89-4975-be16-e978fed8329c. 100%|██████████| 12/12 [00:07<00:00, 1.52it/s] 2020-05-26 14:20:23,341 - up42.project - INFO - Using existing workflow: S1-GRD_SNAP, 1585a205-3907-4905-bd02-057a17c3cf84. 2020-05-26 14:20:24,728 - up42.tools - INFO - Getting blocks name and id, use basic=False for all block details.
{'tiling': 'd350aa0b-ac31-4021-bbe6-fd8da366740a', 'oneatlas-spot-aoiclipped': '0f15e07f-efcc-4598-939b-18aade349c57', 'oneatlas-pleiades-aoiclipped': 'f026874d-e95e-4293-b811-7667130e054d', 'sobloo-s1-grd-fullscene': '4524e2de-c780-488d-9818-fe68dad9f095', 'sobloo-s2-l1c-fullscene': '604988cb-8252-4161-bf28-f6fb63d7371c', 'snap-polarimetric': '320158d6-8f93-4484-a828-e1fb64f677ff', 'sentinelhub-s2-aoiclipped': 'c4758545-4b74-4318-ae1f-d5ba72f234ca', 'sentinelhub-landsat8-aoiclipped': 'e0b133ae-7b9c-435c-99ac-c4527cc8d9cf', 'sobloo-s1-slc-fullscene': 'cf822545-c73c-467b-8f43-5311dbefe03f', 'nasa-modis': '61279eb8-02e1-4b7a-ac3d-1f62d19d3484', 's2-superresolution': '4872fef8-aec8-4dec-adcb-560ee4430a2b', 'oneatlas-pleiades-fullscene': '8487adcd-a4d7-4cb7-b826-75a533e1f330', 'oneatlas-spot-fullscene': 'aa62113f-0dd1-40a3-a004-954c9d087071', 'data-conversion': '470eedda-5f62-433c-8562-98eb8783af87', 'pansharpen': '2f24c662-c129-409f-a7c3-afa16a4c78cb', 'sobloo-s1-grd-aoiclipped': 'a956166f-c0ed-4670-8a43-87bed8d222f3', 'hello-world': '77899bd0-40de-4aa7-b67e-513a5655afcb', 'ndvi': 'aecb81ef-1c92-4b2e-aa25-55ccebe2f90a', 'sobloo-s2-l1c-aoiclipped': 'a2daaab4-196d-4226-a018-a810444dcad1', 'sobloo-s3': 'fee13ec1-a067-4d6a-95dc-a4fef458f4f4', 'sobloo-s5p': 'cba7c59b-548d-48bf-8920-c20d7d316dfd', 'kmeans-clustering': 'adf21e0a-98bf-41a9-a2cc-a59898a461ba', 'vectorising': '295bf286-0748-474f-aa38-c1ac35151204', 'crs-conversion': '18e6772f-cf33-4955-b7e4-61df8a108fd9', 'sharpening': '4ed70368-d4e1-4462-bef6-14e768049471', 'zonal-statistics': 'a5b8b938-6fd6-4bac-92cd-dffd7f3169aa', 'superresolution': '6c914299-7203-40ad-9b89-de0b4e827bd0', 's5p-lvl3': '9a593e06-eca0-49e0-8b8c-6fe95f699f9d', 'hexagon-aerial-30cm': '0b04d9f7-3a8e-4467-9fb8-1e8343c9469a', 'hexagon-aerial-15cm': '36fe7d3a-4671-424b-bd54-918dd21cfde1', 'ship-identification': '20a3bd1e-3f27-40cf-9915-8fa3d5024ade', 'ais-hvp': '7394287a-2458-4204-be62-36b6d264bcfe', 'ais-hvt': '67eb1763-abeb-4188-b135-f6a0d669d759', 'meteomatics': 'ed0beedb-111b-4285-aa2d-f876a4c16a32', 'oneatlas-pleiades-primary': 'd1e5e0de-71fa-4488-9c0e-3f22ac74a2b6', 'tiled-k-means': '45c0284a-5cd7-4bc0-9d6c-db05f7271036', 'data-conversion-netcdf': 'c358d5af-7819-4ecf-b8b3-629d5d3ba319', 'data-conversion-dimap': '25f42430-d108-4ea4-a81d-2c2b3fff7d11', 'aws-s2-l2a': '73f1b9e0-83de-4cbd-addf-5e0d23ed65d4', 'nextmapone-3m': '56a0ac08-69a5-491a-993d-63e3b9da315c', 'nextmapone-6m': 'f87dd9f3-fc14-48b1-b77d-d695fb65fcc4', 'nextmapone-1m': 'bfd43fbc-b662-4874-9147-658a55bf9edc'}
# Fill the workflow with tasks
input_tasks= ['sobloo-s1-grd-fullscene', 'snap-polarimetric']
workflow.add_workflow_tasks(input_tasks=input_tasks)
workflow.get_parameters_info()
2020-05-26 14:21:03,273 - up42.workflow - INFO - Added tasks to workflow: [{'name': 'sobloo-s1-grd-fullscene:1', 'parentName': None, 'blockId': '4524e2de-c780-488d-9818-fe68dad9f095'}, {'name': 'snap-polarimetric:1', 'parentName': 'sobloo-s1-grd-fullscene:1', 'blockId': '320158d6-8f93-4484-a828-e1fb64f677ff'}] 2020-05-26 14:21:04,000 - up42.workflow - INFO - Got 2 tasks/blocks in workflow 1585a205-3907-4905-bd02-057a17c3cf84.
{'sobloo-s1-grd-fullscene:1': {'ids': {'type': 'array', 'default': None}, 'bbox': {'type': 'array', 'default': None}, 'time': {'type': 'dateRange', 'default': '2018-01-01T00:00:00+00:00/2020-12-31T23:59:59+00:00'}, 'limit': {'type': 'integer', 'default': 1, 'minimum': 1}, 'contains': {'type': 'geometry'}, 'intersects': {'type': 'geometry'}, 'time_series': {'type': 'array', 'default': None}, 'orbit_direction': {'type': 'string', 'default': None}, 'acquisition_mode': {'type': 'string', 'default': None}}, 'snap-polarimetric:1': {'bbox': {'type': 'array', 'default': None}, 'mask': {'type': 'array', 'items': {'enum': ['land', 'sea'], 'type': 'string'}, 'default': None}, 'contains': {'type': 'geometry', 'default': None}, 'intersects': {'type': 'geometry', 'default': None}, 'clip_to_aoi': {'type': 'boolean', 'default': False}, 'tcorrection': {'type': 'boolean', 'default': True}, 'linear_to_db': {'type': 'boolean', 'default': True}, 'polarisations': {'type': 'array', 'items': {'enum': ['VV', 'VH'], 'type': 'string'}, 'default': ['VV'], 'required': False, 'description': 'Requested polarisations for the output'}, 'speckle_filter': {'type': 'boolean', 'default': True}, 'calibration_band': {'type': 'array', 'default': ['sigma']}}}
The S1 GRD block always delivers the complete images as they are delivered in SAFE format which cannot be clipped. The image can be clipped though by the SNAP polarimetric processing blocks. For this it is necessary to supply the same geometry (in this case a bbox) to that block as well.
input_parameters = {
"sobloo-s1-grd-fullscene:1": {
"bbox": [13.371037, 52.512799, 13.382624, 52.518747],
"ids": None,
"time": "2018-01-01T00:00:00+00:00/2020-12-31T23:59:59+00:00",
"limit": 1,
"zoom_level": 14,
},
"snap-polarimetric:1": {
"bbox": [13.371037, 52.512799, 13.382624, 52.518747],
"mask": None,
"contains": None,
"intersects": None,
"clip_to_aoi": True,
"tcorrection": True,
"linear_to_db": True,
"polarisations": [
"VV"
],
"speckle_filter": True,
"calibration_band": [
"sigma"
]
}
}
Run a test job to query data availability and check the configuration.
test_job = workflow.test_job(input_parameters=input_parameters, track_status=True)
test_results = test_job.get_results_json()
print(test_results)
2020-05-26 14:21:14,155 - up42.workflow - INFO - +++++++++++++++++++++++++++++++++ 2020-05-26 14:21:14,157 - up42.workflow - INFO - Running this job as Test Query... 2020-05-26 14:21:14,157 - up42.workflow - INFO - +++++++++++++++++++++++++++++++++ 2020-05-26 14:21:14,158 - up42.workflow - INFO - Selected input_parameters: {'sobloo-s1-grd-fullscene:1': {'bbox': [13.371037, 52.512799, 13.382624, 52.518747], 'ids': None, 'time': '2018-01-01T00:00:00+00:00/2020-12-31T23:59:59+00:00', 'limit': 1, 'zoom_level': 14}, 'snap-polarimetric:1': {'bbox': [13.371037, 52.512799, 13.382624, 52.518747], 'mask': None, 'contains': None, 'intersects': None, 'clip_to_aoi': True, 'tcorrection': True, 'linear_to_db': True, 'polarisations': ['VV'], 'speckle_filter': True, 'calibration_band': ['sigma']}, 'config': {'mode': 'DRY_RUN'}}. 2020-05-26 14:21:15,659 - up42.workflow - INFO - Created and running new job: 28c72c74-a5dc-46e3-bb70-f6efad3f910c. 2020-05-26 14:21:16,144 - up42.job - INFO - Tracking job status continuously, reporting every 30 seconds... 2020-05-26 14:21:39,235 - up42.job - INFO - Job finished successfully! - 28c72c74-a5dc-46e3-bb70-f6efad3f910c 2020-05-26 14:21:44,956 - up42.job - INFO - Retrieved 1 features.
{'type': 'FeatureCollection', 'features': [{'type': 'Feature', 'bbox': [12.306028, 52.168652, 16.66515, 54.078037], 'id': '5a0bcc6b-8316-45c9-9106-8de8e27e5753', 'geometry': {'type': 'Polygon', 'coordinates': [[[16.151102, 52.168652], [16.66515, 53.660217], [12.685362, 54.078037], [12.306028, 52.5835], [16.151102, 52.168652]]]}, 'properties': {'attachments': [], 'visibility': {'deleted': False}, 'illumination': {}, 'production': {'levelCode': 'L1', 'ongoing': False, 'timeliness': 'Fast-24h'}, 'archive': {'offLine': False, 'filename': 'S1A_IW_GRDH_1SDV_20200525T051659_20200525T051724_032717_03CA2F_DB50.SAFE', 'size': 1689, 'format': 'SAFE', 'onLine': False}, 'spatialCoverage': {'verticality': {}, 'geometry': {'geographicBoundingPolygon': {'coordinates': [[[16.151102, 52.168652], [16.66515, 53.660217], [12.685362, 54.078037], [12.306028, 52.5835], [16.151102, 52.168652]]], 'type': 'Polygon'}, 'global': False, 'centerPoint': {'lon': 14.45556570632881, 'lat': 53.12652491313295}}}, 'quality': {'qualified': False}, 'target': {}, 'timeStamp': 1590383819131, 'uid': '5a0bcc6b-8316-45c9-9106-8de8e27e5753', 'identification': {'profile': 'Image', 'externalId': 'S1A_IW_GRDH_1SDV_20200525T051659_20200525T051724_032717_03CA2F_DB50', 'collection': 'Sentinel-1', 'type': 'GRD', 'dataset': {}}, 'transmission': {}, 'contentDescription': {}, 'provider': {}, 'acquisition': {'endViewingDate': 1590383844129, 'mission': 'Sentinel-1', 'missionId': 'A', 'missionCode': 'S1A', 'beginViewingDate': 1590383819131, 'missionName': 'Sentinel-1A', 'polarization': 'VV VH', 'type': 'NOMINAL', 'sensorMode': 'IW', 'sensorId': 'SAR-C SAR'}, 'orbit': {'relativeNumber': 95, 'lastRelativeNumber': 95, 'lastNumber': 32717, 'direction': 'DESCENDING'}, 'state': {'resources': {'thumbnail': True, 'quicklook': True}, 'services': {'wmts': False, 'download': 'internal', 'wcs': False, 'wms': False}, 'insertionDate': 1590395388355}, 'attitude': {}}}]}
job = workflow.run_job(input_parameters=input_parameters, track_status=True)
2020-05-26 14:21:44,963 - up42.workflow - INFO - Selected input_parameters: {'sobloo-s1-grd-fullscene:1': {'bbox': [13.371037, 52.512799, 13.382624, 52.518747], 'ids': None, 'time': '2018-01-01T00:00:00+00:00/2020-12-31T23:59:59+00:00', 'limit': 1, 'zoom_level': 14}, 'snap-polarimetric:1': {'bbox': [13.371037, 52.512799, 13.382624, 52.518747], 'mask': None, 'contains': None, 'intersects': None, 'clip_to_aoi': True, 'tcorrection': True, 'linear_to_db': True, 'polarisations': ['VV'], 'speckle_filter': True, 'calibration_band': ['sigma']}}. 2020-05-26 14:21:46,310 - up42.workflow - INFO - Created and running new job: 2e0e9cf0-8d80-439a-bc50-179543641ea9. 2020-05-26 14:21:46,734 - up42.job - INFO - Tracking job status continuously, reporting every 30 seconds... 2020-05-26 14:22:21,006 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9 2020-05-26 14:23:06,480 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9 2020-05-26 14:23:40,185 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9 2020-05-26 14:24:13,848 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9 2020-05-26 14:24:47,455 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9 2020-05-26 14:25:21,147 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9 2020-05-26 14:25:54,825 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9 2020-05-26 14:26:28,527 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9 2020-05-26 14:27:02,484 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9 2020-05-26 14:27:36,284 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9 2020-05-26 14:28:09,935 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9 2020-05-26 14:28:42,827 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9 2020-05-26 14:29:15,409 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9 2020-05-26 14:29:48,015 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9 2020-05-26 14:30:20,694 - up42.job - INFO - Job finished successfully! - 2e0e9cf0-8d80-439a-bc50-179543641ea9
# Download results:
job.download_results()
# Visualize downloaded results
job.plot_results()
The output of the workflow is a GeoTIFF file which can be processed by different algorithms. It is for example possible to apply raster tiling and then do some machine learning based analysis at the end.