Freva offers the possibility to share custom datasets with other users by making it searchable via freva.databrowser. With help of the UserData class users can add their own data to the central data location, (re)-index or delete data in the databrowser.
Note: Any data that has been added by users will be assigned a special project name: project=user-$USER.
To be able to add data to the databrowser the file names must follow a strict standard and the files must reside in a specific location. This add method takes care about the correct file naming and location. No pre-requirements other than the file has to be a valid netCDF or grib file are assumed. In other words this method places the user data with the correct naming structure to the correct location.
Suppose you’ve gotten data from somewhere and want to add this data into the databrowser to make it accessible to others. In this specific example we assume that you have stored your original data in the /tmp/my_awesome_data folder. E.g /tmp/my_awesome_data/outfile_0.nc...tmp/my_awesome_data/outfile_9.nc The routine will try to gather all necessary metadata from the files. You’ll have to provide additional metadata if mandatory keywords are missing. To make the routine work you’ll have to provide the institute, model and experiment keywords:
from freva import UserData, databrowser
Let's create an instance of the UserData class.
user_data = UserData()
The add method can add data into the users data directory. The location of this data directory is given by the system and can be queried by accessing the user_dir property:
# Get the location of the user data
print(user_data.user_dir)
Let's inspect the help of the add method:
help(user_data.add)
# You can also provide wild cards to search for data
user_data.add("eur-11b", "/tmp/my_awesome_data/outfile_?.nc",
institute="clex", model="UM-RA2T",
experiment="Bias-correct")
# Check the databrowser if the data has been added
Let's check if the data has been added to the data browser
for file in databrowser(experiment="bias*"):
print(file)
By default the data is copied. By using the how keyword you can also link or move the data.
The delete removes entries from the databrowser and if necessary existing files from the central user data location. Let's inspect the help of the user delete method first:
help(user_data.delete)
user_data.delete(user_data.user_dir)
Check the databrowser if the data has been removed:
databrowser(experiment="bias*", count=True)
Using the index method the databrowser can be updated with existing user data. For example, if data has been removed from the databrowser it can be re-added. Again, inspect the help first:
help(user_data.index)
user_data.index()