AERONET¶

About¶

The AERONET (AErosol RObotic NETwork) project is a federation of ground-based remote sensing aerosol networks established by NASA and LOA-PHOTONS (CNRS) and is greatly expanded by collaborators from national agencies, institutes, universities, individual scientists, and partners. The program provides a long-term (more than 25 years), continuous and readily accessible public domain database of aerosol optical, microphysical and radiative properties for aerosol research and characterization, validation of satellite retrievals, and synergism with other databases. The network imposes standardization of instruments, calibration, processing and distribution.

AERONET collaboration provides globally distributed observations of spectral aerosol optical Depth (AOD), inversion products, and precipitable water in diverse aerosol regimes. Aerosol optical depth data are computed for three data quality levels: Level 1.0 (unscreened), Level 1.5 (cloud-screened), and Level 2.0 (cloud screened and quality-assured). Inversions, precipitable water, and other AOD-dependent products are derived from these levels and may implement additional quality checks.

You can see an overview of all available AERONET Site Names here.

Basic Facts¶

Spatial coverage: Observation stations worldwide
Temporal resolution: sub-daily and daily
Temporal coverage: since 1993
Data format: txt
Versions: Level 1.0 (unscreened), Level 1.5 (cloud-screened), Level 2.0 (cloud screened and quality-assured)

How to access the data¶

AERONET offers a web service which allows you to request and save aeronet data via wget, which is a command to download files from the internet. The AERONET web service endpoint is available under https://aeronet.gsfc.nasa.gov/cgi-bin/print_web_data_v3 and a detailed documentation of how to construct requests can be found here.

The first part of this notebook (1 - Download AERONET data for a specific station and time period) shows you how to request AERONET data with a wget command.

Module outline¶

• 1 - Download AERONET data for a specific station and time period
• 2 - Read the observation data with pandas
• 3 - Visualize all points of AERONET Aerosol Optical Depth in Palma de Mallorca for February 2021
• 4 - Load and visualize daily Aerosol Optical Depth aggregates together with the Angstrom Exponent

Load libraries¶

1. Download AERONET data for a specific station and time period¶

AERONET offers a web service which allows you to request and save aeronet data via wget, which is a command to download files from the internet. The AERONET web service endpoint is available under https://aeronet.gsfc.nasa.gov/cgi-bin/print_web_data_v3 and a detailed documentation of how to construct requests can be found here.

An example request from the website looks like this:

wget --no-check-certificate -q -O test.out "https://aeronet.gsfc.nasa.gov/cgi-bin/print_web_data_v3?site=Cart_Site&year=2000&month=6&day=1&year2=2000&month2=6&day2=14&AOD15=1&AVG=10"

In this request, you use wget to download data from the webservice. You tailor your request with a set of keywords which you concatenate to the service endpoint with &. However, constructing such requests manually can be quite cumbersome. For this reason, we will show you an approach how you can dynamically generate and request AERONET data from the web service with Python.

To run a wget request like the one above in a Jupyter Notebook code cell, you should use !wget before adding the rest of the request. For example (commented out):

You do not need to use the ! if you are using the method shown below where you call the download method from the wget library.

As a first step, let us create a Python dictionary in which we store all the parameters we would like to use for the request as dictionary keys. You can initiate a dictionary with curled brackets {}. Below, we specify the following parameters:

• endpoint: Endpoint of the AERONET web service
• station: Name of the AERONET station
• year: year 1 of interest
• month: month 1 of interest
• day: day 1 of interest
• year2: year 2 of interest
• month2: month 2 of interest
• day2: day 2 of interest
• AOD15: data type, other options include AOD10, AOD20, SDA10,SDA15 etc.
• AVG: data format, AVG=10 - all points, AVG=20 - daily averages

Please note, that there are additional parameters that can be set, e.g. hour. The keywords below are those we will need for requesting all data points of Aerosol Optical Depth Level 1.5 data for the station Palma de Mallorca for February 2021.

In a next step, we construct the final string for the wget request with the format function. You construct a string by adding the dictionary keys in curled brackets. At the end of the string, you provide the dictionary key informatoin to the string with the format() function. A print of the resulting url shows, that the format function replaced the information in the curled brackets with the data in the dictionary.

This formula is created for downloading the data type AOD15. If you want to download another data type you should replace both occurences of AOD15 below with another data type.

Now we are ready to request the data with the function download() from the wget Python library. You have to pass to the function the constructed url above together with a file path of where the downloaded that shall be stored. Let us store the data as txt file in the same folder as this notebook as a demonstration of how the downloading works.

Let us repeat the data request and let us also request the daily average of Aerosol Optical Depth for February 2021 for the station Palma de Mallorca. The parameter you have to change is AVG. By setting 20, you indicate the request that you are interested in daily averages. You can make the required changes in the dictionary above and re-run the data request. Make sure to also change the name of the output file to e.g. 202102_palma_aod15_20.txt.

2. Read the observation data with pandas¶

The next step is now to read the downloaded txt file. Let us start with the file of all station measurements for the station Palma de Mallorca in February 2021. The file has the name 202102_palma_aod15_10.txt. You can read txt files with the function read_table from the Python library pandas. If you inspect the txt file before (you can simply open it), you see that the first few lines contain information we do not need in the pandas dataframe. For this reason, we can set additonal keyword arguments that allow us to specify the columns and rows of interest:

• delimiter: specify the delimiter in the text file, e.g. comma
• header: specify the index of the row that shall be set as header.
• index_col: specify the index of the column that shall be set as index

You see below that the resulting dataframe has 258 rows and 113 columns.

In the dataframe above, you see that missing data entries are filled with -999.0. Let us replace those as NaN. You can use the function replace() to do so. The resulting dataframe has now NaN (not a number) for all entries with missing data. This replacement facilitates the plotting of the data.

To get a complete overview of all column information, you can print a list of all column headers. You can do this with the function columns.to_list().

If you would like to narrow down the selection of the dataframe, you can apply the filter function and filter the dataframe based on a given regular expression. For example, let us only select the columns that contain Angstrom_Exponent information and additionally only select the first 100 rows.

This selection shows you that the station does not provide measurements for each day. The first measurements are for 2 February and the measurements are again continued on 6 February 2021.

3. Visualize all points of AERONET Aerosol Optical Depth in Palma de Mallorca for February 2021¶

The next step is to visualize all points of Aerosol Optical Depth at different wavelengths in Palma de Mallorca for February 2021. We specifically want to plot the AOD information for which we have measurements, which are the following: AOD_1640nm, AOD_1020nm, AOD_870nm, AOD_675nm, AOD_500nm, AOD_440nm, AOD_380nm and AOD_340nm.

You can use the built-in plot() function of the pandas library to define a line plot. With the filter function, you can select the dataframe columns you wish to visualize. The visualisation code below consists of five main parts:

• Initiate a matplotlib figure
• Define a line plot with the built-in plot function of the pandas library
• Set title and axes label information
• Format axes ticks
• Add additional features, such as a grid or legend

Below, you see that the Aerosol Optical Depth increases between the 5th and 7th February 2021 as well as around the 21st February 2021.

4. Load and visualize daily Aerosol Optical Depth aggregates together with the Angstrom Exponent¶

In a next step, we would like to visualize the Aerosol Optical Depth information together with the Angstrom Exponent in the spectral interval of 440-480nm. The Angstrom Exponent is often used as a qualitative indicator of aerosol particle size. To visualize both variables together, we load the daily averaged AERONET information, which we stored as txt file with the name 202102_palma_aod15_20.txt. You can use again the function read_table from the pandas library to read the txt file.

Compared to the all point information, you see that the dataframe with the daily aggregated values has only 15 rows and 81 columns. As before, we only have daily aggregated values for days where the station had measurements.

The next step is again to replace the value for missing data (-999.0) as NaN. You can do this with the function replace().

As a final step, we can use again the plotting code from above and visualize the daily aggregated Aerosol Optical Depth information. Additionally to the AOD values as line plot, we add the 440-870_Angstrom_Exponent column as bar plot. You can do this in a similar way. You use the pandas built-in plot() function and specify the keyword argument kind='bar', which indicates the function to plot a bar graph.

Above, you see elevated AOD values and low values for the Angstrom exponent for the Saharan dust event between 5 to 7 February 2021. Additionally, the time-series shows that there might have been a second dust event towards the end of February 2021, that had even a stronger impact on the station in Palma de Mallorca. Let us inspect the daily aggregated values of AOD of another station, e.g. Ispra in Northern Italy for February 2021.

5. Load and visualize Aerosol Optical Depth and Angstrom Exponent for station Ispra in February 2021¶

Let us load the daily aggregated Aerosol Optical Depth values for station Ispra in north Italy for February 2021.

And now, let us use again the plotting code from above and visualize the daily aggregated Aerosol Optical Depth information for Ispra in February 2021. Additionally to the AOD values as line plot, we add the 440-870_Angstrom_Exponent column as bar plot. You see, that Ispra has elevated AOD measurements for a second Saharan dust event, which took place between 21 and 24 February 2021.

References¶

• Giles, D. M., Sinyuk, A., Sorokin, M. G., Schafer, J. S., Smirnov, A., Slutsker, I., Eck, T. F., Holben, B. N., Lewis, J. R., Campbell, J. R., Welton, E. J., Korkin, S. V., and Lyapustin, A. I. (2019): Advancements in the Aerosol Robotic Network (AERONET) Version 3 database – automated near-real-time quality control algorithm with improved cloud screening for Sun photometer aerosol optical depth (AOD) measurements, Atmos. Meas. Tech., 12, 169-209, https://doi.org/10.5194/amt-12-169-2019.

• Holben B.N., T.F.Eck, I.Slutsker, D.Tanre, J.P.Buis, A.Setzer, E.Vermote, J.A.Reagan, Y.Kaufman, T.Nakajima, F.Lavenu, I.Jankowiak, and A.Smirnov, 1998: AERONET - A federated instrument network and data archive for aerosol characterization, Rem. Sens. Environ., 66, 1-16.

This project is licensed under GNU General Public License v3.0 only and is developed under a Copernicus contract.

EUMETSAT Training | Contact the training team