import pyaurorax
import datetime
import pprint
import pandas as pd
aurorax = pyaurorax.PyAuroraX()
# set values
start = datetime.datetime(2019, 1, 1, 0, 0, 0)
end = datetime.datetime(2019, 1, 1, 23, 59, 59)
programs = ["swarm"]
platforms = ["swarma"]
instrument_types = ["footprint"]
# perform search
s = aurorax.search.ephemeris.search(start, end, programs=programs, platforms=platforms, instrument_types=instrument_types, verbose=True)
[2024-06-17 19:58:39.263277] Search object created [2024-06-17 19:58:39.293216] Request submitted [2024-06-17 19:58:39.293216] Request ID: 537bc2b4-8996-4292-bf7b-fda87acf6ca5 [2024-06-17 19:58:39.293216] Request details available at: https://api.aurorax.space/api/v1/ephemeris/requests/537bc2b4-8996-4292-bf7b-fda87acf6ca5 [2024-06-17 19:58:39.293216] Waiting for data ... [2024-06-17 19:58:40.382840] Checking for data ... [2024-06-17 19:58:40.465122] Data is now available [2024-06-17 19:58:40.465122] Retrieving data ... [2024-06-17 19:58:41.146227] Retrieved 8.3 MB of data containing 1440 records
# output data as a pandas dataframe
ephemeris_data = [e.__dict__ for e in s.data]
df = pd.DataFrame(ephemeris_data)
df.sort_values("epoch") # sort by the timestamp
data_source | epoch | location_geo | nbtrace | sbtrace | location_gsm | metadata | |
---|---|---|---|---|---|---|---|
0 | DataSource(identifier=3, program='swarm', plat... | 2019-01-01 00:00:00 | Location(lat=-17.019026, lon=-136.0207) | Location(lat=14.933351, lon=-130.16835) | Location(lat=-21.815926, lon=-137.23001) | Location(lat=6.7606134, lon=40.40137) | {'state': 'definitive', 'tii_on': True, 'nbtra... |
1 | DataSource(identifier=3, program='swarm', plat... | 2019-01-01 00:01:00 | Location(lat=-13.168878, lon=-136.07999) | Location(lat=12.554154, lon=-131.52226) | Location(lat=-19.18973, lon=-137.46463) | Location(lat=10.541506, lon=41.12732) | {'state': 'definitive', 'tii_on': True, 'nbtra... |
2 | DataSource(identifier=3, program='swarm', plat... | 2019-01-01 00:02:00 | Location(lat=-9.317859, lon=-136.14517) | Location(lat=10.6922245, lon=-132.69847) | Location(lat=-17.108786, lon=-137.80627) | Location(lat=14.321112, lon=41.87248) | {'state': 'definitive', 'tii_on': True, 'nbtra... |
3 | DataSource(identifier=3, program='swarm', plat... | 2019-01-01 00:03:00 | Location(lat=-5.466018, lon=-136.21439) | Location(lat=9.6191, lon=-133.67313) | Location(lat=-15.831487, lon=-138.30301) | Location(lat=18.098713, lon=42.64437) | {'state': 'definitive', 'tii_on': True, 'nbtra... |
4 | DataSource(identifier=3, program='swarm', plat... | 2019-01-01 00:04:00 | Location(lat=-1.6134088, lon=-136.2859) | Location(lat=9.573847, lon=-134.42802) | Location(lat=-15.586076, lon=-139.00354) | Location(lat=21.873484, lon=43.451508) | {'state': 'definitive', 'tii_on': True, 'nbtra... |
... | ... | ... | ... | ... | ... | ... | ... |
1435 | DataSource(identifier=3, program='swarm', plat... | 2019-01-01 23:55:00 | Location(lat=63.118088, lon=39.44719) | Location(lat=63.825092, lon=39.852196) | Location(lat=-52.62199, lon=65.487434) | Location(lat=38.491505, lon=-150.19873) | {'state': 'definitive', 'nbtrace_region': 'nor... |
1436 | DataSource(identifier=3, program='swarm', plat... | 2019-01-01 23:56:00 | Location(lat=59.271484, lon=39.970604) | Location(lat=60.096794, lon=40.335415) | Location(lat=-48.855095, lon=61.44911) | Location(lat=34.721504, lon=-149.17068) | {'state': 'definitive', 'nbtrace_region': 'nor... |
1437 | DataSource(identifier=3, program='swarm', plat... | 2019-01-01 23:57:00 | Location(lat=55.42148, lon=40.336536) | Location(lat=56.37933, lon=40.665756) | Location(lat=-44.895798, lon=57.84364) | Location(lat=30.944128, lon=-148.23186) | {'state': 'definitive', 'nbtrace_region': 'nor... |
1438 | DataSource(identifier=3, program='swarm', plat... | 2019-01-01 23:58:00 | Location(lat=51.569214, lon=40.592674) | Location(lat=52.677044, lon=40.891216) | Location(lat=-40.812534, lon=54.591457) | Location(lat=27.161028, lon=-147.36389) | {'state': 'definitive', 'nbtrace_region': 'nor... |
1439 | DataSource(identifier=3, program='swarm', plat... | 2019-01-01 23:59:00 | Location(lat=47.715435, lon=40.76942) | Location(lat=48.9952, lon=41.04236) | Location(lat=-36.66003, lon=51.658134) | Location(lat=23.373522, lon=-146.55217) | {'state': 'definitive', 'nbtrace_region': 'nor... |
1440 rows × 7 columns
Under the hood, the AuroraX API performs an ephemeris search asynchronously. Note that this does not mean that it can be done using a Python async method; it means that PyAuroraX does more than just a single HTTP request against the AuroraX API when doing a search. With the API operating this way, it adds some more complexity within PyAuroraX but also opens the search up to some very important capabilities. The main capability enabled by this architecture is the ablity to perform queries for a large timeframe, and/or for a large number of data sources. Queries like this can easily take several minutes, and can return hundreds of megabytes - or eve gigabytes - of data. A conventional HTTP request would normally timeout because of this, ultimately failing to complete the search.
Instead of using the pyaurorax.ephemeris.search
method which wraps all logic into an easy function, you can also perform an ephemeris search step-by-step if you want more control over the process. Below, we do a search in this manner.
# set up search parameters
start = datetime.datetime(2019, 1, 1, 0, 0, 0)
end = datetime.datetime(2019, 1, 1, 23, 59, 59)
programs = ["swarm"]
platforms = ["swarma"]
instrument_types = ["footprint"]
# create the Search object
s = pyaurorax.search.EphemerisSearch(aurorax, start, end, programs=programs, platforms=platforms, instrument_types=instrument_types)
print(s)
EphemerisSearch(executed=False, completed=False, request_id='')
# execute the search
s.execute()
print(s)
EphemerisSearch(executed=True, completed=False, request_id='01bacc43-70a2-4efd-b80b-1b49788b9c21')
# get request status
s.update_status()
pprint.pprint(s.status)
{'logs': [{'level': 'debug', 'summary': 'Search request arrived and saved', 'timestamp': '2024-06-18T01:59:08.188382'}, {'level': 'debug', 'summary': 'Starting 01bacc43-70a2-4efd-b80b-1b49788b9c21 for query ' 'Find ephemeris for ((program in (swarm) AND platform in ' '(swarma) AND instrument_type in (footprint) filtered by ' 'metadata ()) AND ephemeris_metadata_filters []) AND ' 'epoch between 2019-01-01T00:00 and 2019-01-01T23:59:59 ' 'UTC', 'timestamp': '2024-06-18T01:59:08.191365'}, {'level': 'info', 'summary': 'Finished query in 0.02 seconds.', 'timestamp': '2024-06-18T01:59:08.211663'}, {'level': 'debug', 'summary': 'Starting to write data to file', 'timestamp': '2024-06-18T01:59:08.215357'}, {'level': 'info', 'summary': 'Finished writing data, found 1440 records in 0.42 ' 'seconds.', 'timestamp': '2024-06-18T01:59:08.634'}, {'level': 'debug', 'summary': 'File size is 8333523 bytes.', 'timestamp': '2024-06-18T01:59:08.637182'}, {'level': 'info', 'summary': 'Finished search in 0.45 seconds and found 1440 records.', 'timestamp': '2024-06-18T01:59:08.640955'}], 'search_request': {'query': {'data_sources': {'ephemeris_metadata_filters': {'expressions': [], 'logical_operator': 'AND'}, 'instrument_types': ['footprint'], 'platforms': ['swarma'], 'programs': ['swarm']}, 'end': '2019-01-01T23:59:59', 'request_id': None, 'start': '2019-01-01T00:00:00'}, 'request_id': '01bacc43-70a2-4efd-b80b-1b49788b9c21', 'request_type': 'ephemeris', 'requested': '2024-06-18T01:59:08.183911'}, 'search_result': {'completed_timestamp': '2024-06-18T01:59:08.639167', 'data_uri': '/api/v1/ephemeris/requests/01bacc43-70a2-4efd-b80b-1b49788b9c21/data', 'error_condition': False, 'file_size': 8333523, 'query_duration': 448, 'result_count': 1440, 'result_file_deleted_timestamp': None}}
# view just the logs for the request (update the status beforehand, but we do that in the above cell)
pprint.pprint(s.logs)
[{'level': 'debug', 'summary': 'Search request arrived and saved', 'timestamp': '2024-06-18T01:59:08.188382'}, {'level': 'debug', 'summary': 'Starting 01bacc43-70a2-4efd-b80b-1b49788b9c21 for query Find ' 'ephemeris for ((program in (swarm) AND platform in (swarma) AND ' 'instrument_type in (footprint) filtered by metadata ()) AND ' 'ephemeris_metadata_filters []) AND epoch between ' '2019-01-01T00:00 and 2019-01-01T23:59:59 UTC', 'timestamp': '2024-06-18T01:59:08.191365'}, {'level': 'info', 'summary': 'Finished query in 0.02 seconds.', 'timestamp': '2024-06-18T01:59:08.211663'}, {'level': 'debug', 'summary': 'Starting to write data to file', 'timestamp': '2024-06-18T01:59:08.215357'}, {'level': 'info', 'summary': 'Finished writing data, found 1440 records in 0.42 seconds.', 'timestamp': '2024-06-18T01:59:08.634'}, {'level': 'debug', 'summary': 'File size is 8333523 bytes.', 'timestamp': '2024-06-18T01:59:08.637182'}, {'level': 'info', 'summary': 'Finished search in 0.45 seconds and found 1440 records.', 'timestamp': '2024-06-18T01:59:08.640955'}]
# wait for the data
s.wait()
s.update_status()
# get data
s.get_data()
print(s.data[0])
# show data as pandas dataframe
ephemeris_data = [e.__dict__ for e in s.data]
df = pd.DataFrame(ephemeris_data)
df.sort_values("epoch") # sort by the timestamp
EphemerisData(epoch=datetime.datetime(2019, 1, 1, 0, 0), location_geo=Location(lat=-17.019026, lon=-136.0207), location_gsm=Location(lat=6.7606134, lon=40.40137), nbtrace=Location(lat=14.933351, lon=-130.16835), sbtrace=Location(lat=-21.815926, lon=-137.23001), metadata={'state': 'definitiv...}, data_source=DataSource(...))
data_source | epoch | location_geo | nbtrace | sbtrace | location_gsm | metadata | |
---|---|---|---|---|---|---|---|
0 | DataSource(identifier=3, program='swarm', plat... | 2019-01-01 00:00:00 | Location(lat=-17.019026, lon=-136.0207) | Location(lat=14.933351, lon=-130.16835) | Location(lat=-21.815926, lon=-137.23001) | Location(lat=6.7606134, lon=40.40137) | {'state': 'definitive', 'tii_on': True, 'nbtra... |
1 | DataSource(identifier=3, program='swarm', plat... | 2019-01-01 00:01:00 | Location(lat=-13.168878, lon=-136.07999) | Location(lat=12.554154, lon=-131.52226) | Location(lat=-19.18973, lon=-137.46463) | Location(lat=10.541506, lon=41.12732) | {'state': 'definitive', 'tii_on': True, 'nbtra... |
2 | DataSource(identifier=3, program='swarm', plat... | 2019-01-01 00:02:00 | Location(lat=-9.317859, lon=-136.14517) | Location(lat=10.6922245, lon=-132.69847) | Location(lat=-17.108786, lon=-137.80627) | Location(lat=14.321112, lon=41.87248) | {'state': 'definitive', 'tii_on': True, 'nbtra... |
3 | DataSource(identifier=3, program='swarm', plat... | 2019-01-01 00:03:00 | Location(lat=-5.466018, lon=-136.21439) | Location(lat=9.6191, lon=-133.67313) | Location(lat=-15.831487, lon=-138.30301) | Location(lat=18.098713, lon=42.64437) | {'state': 'definitive', 'tii_on': True, 'nbtra... |
4 | DataSource(identifier=3, program='swarm', plat... | 2019-01-01 00:04:00 | Location(lat=-1.6134088, lon=-136.2859) | Location(lat=9.573847, lon=-134.42802) | Location(lat=-15.586076, lon=-139.00354) | Location(lat=21.873484, lon=43.451508) | {'state': 'definitive', 'tii_on': True, 'nbtra... |
... | ... | ... | ... | ... | ... | ... | ... |
1435 | DataSource(identifier=3, program='swarm', plat... | 2019-01-01 23:55:00 | Location(lat=63.118088, lon=39.44719) | Location(lat=63.825092, lon=39.852196) | Location(lat=-52.62199, lon=65.487434) | Location(lat=38.491505, lon=-150.19873) | {'state': 'definitive', 'nbtrace_region': 'nor... |
1436 | DataSource(identifier=3, program='swarm', plat... | 2019-01-01 23:56:00 | Location(lat=59.271484, lon=39.970604) | Location(lat=60.096794, lon=40.335415) | Location(lat=-48.855095, lon=61.44911) | Location(lat=34.721504, lon=-149.17068) | {'state': 'definitive', 'nbtrace_region': 'nor... |
1437 | DataSource(identifier=3, program='swarm', plat... | 2019-01-01 23:57:00 | Location(lat=55.42148, lon=40.336536) | Location(lat=56.37933, lon=40.665756) | Location(lat=-44.895798, lon=57.84364) | Location(lat=30.944128, lon=-148.23186) | {'state': 'definitive', 'nbtrace_region': 'nor... |
1438 | DataSource(identifier=3, program='swarm', plat... | 2019-01-01 23:58:00 | Location(lat=51.569214, lon=40.592674) | Location(lat=52.677044, lon=40.891216) | Location(lat=-40.812534, lon=54.591457) | Location(lat=27.161028, lon=-147.36389) | {'state': 'definitive', 'nbtrace_region': 'nor... |
1439 | DataSource(identifier=3, program='swarm', plat... | 2019-01-01 23:59:00 | Location(lat=47.715435, lon=40.76942) | Location(lat=48.9952, lon=41.04236) | Location(lat=-36.66003, lon=51.658134) | Location(lat=23.373522, lon=-146.55217) | {'state': 'definitive', 'nbtrace_region': 'nor... |
1440 rows × 7 columns