Using the BISON API¶

The USGS provides an API for accessing species observation data. https://bison.usgs.gov/doc/api.jsp

This API is much better documented than the NWIS API, and we'll use it to dig a bit deeper into how the requests package can faciliate data access via APIs.

We'll begin by replicating the example API call they show on their web page:

https://bison.usgs.gov/api/search.json?species=Bison bison&type=scientific_name&start=0&count=1

In [ ]:

#First, import the wonderful requests module
import requests

Now, we'll deconstruct the example URL into the service URL and parameters, saving the paramters as a dictionary. Note we are just providing a few of the parameters available through the API. We could add more search criteria if we wanted, but for now we just want to grab the first 500 Bison records.

In [ ]:

# Construct the service URL as two components: the service URL and the request parameters
url = 'http://bison.usgs.gov/api/search.json'
params = {'species':'Bison bison',
          'type':'scientific_name',
          'start':'0',
          'count':'500'
         }

With the components set as variables, we use the requests.get() function to send our request off to the server at the address provided, storing the servers response as a variable called response.

In [ ]:

#Send the request to the server and store the response as a variable
response = requests.get(url,params)

This response object contains a number of properties and methods. Let's have a look at the reponse in raw text format.

In [ ]:

#View the reponse in text format
print(response.text)

Yikes, that's much less readable than the NWIS output!

Well, that's because the response from the BISON server is in JSON format. JSON, short for JavaScript Object Notation, is a text document that stores information in key:value pairs, much like a Python dictionary. Still, it's a raw text object, but one that we convert into a Python dictionary using requests's json() function to convert the servers response into a Python dictionary.

In [ ]:

#Convert the response 
data = response.json()
type(data)

Ok, if it's a dictionary, what are it's keys?

In [ ]:

#List the keys in the returned JSON object
data.keys()

What are the values linked with the 'data' key?

In [ ]:

#Show the value associated with the `data` key
data['data']

Oh, it's a list of occurrences! Let's examine the first one...

In [ ]:

#Display the first "data" value
data['data'][0]

We see it's a dictionary too! Let's list the decimalLatitude item value...

In [ ]:

#We can get the latitude of the record from it's `decimalLatitude` key
data['data'][0]['decimalLatitude']

► So we see the Bison observations are stored as list of dictionaries which are accessed within the data key in the results dictionary generated from the JSON response to our API request. (Phew!)

With a bit more code we can loop through all the data records and print out the lat and long coordinates...

In [ ]:

#Loop thorough each observation and print the lat and long values
for observation in data['data']:
    print (observation['decimalLatitude'],observation['decimalLongitude'])

► If the above throws an error, can you debug it? HINT: the `geo` tag indicates whether coordinate info exist for the record...


#Loop thorough each observation and print the lat and long values
for observation in data['data']:
    if(observation['geo'] == 'Yes'):
        print (observation['decimalLatitude'],observation['decimalLongitude'])

In [ ]:

#Loop thorough each observation and print the lat and long values
for observation in data['data']:
    if(observation['geo'] == 'Yes'):
        print (observation['decimalLatitude'],observation['decimalLongitude'])

Using Pandas to streamline the process...¶

Pandas can create a dataframe directly from dictionary values.

In [ ]:

import pandas as pd
df = pd.DataFrame(data['data'])
df.head()

So now we can use our Panda's know-how to do some nifty analyses, including subsetting records for a specific provider.

First we'll get a list of unique providers found in the data

In [ ]:

#Generate a list of providers
df.provider.unique()

Now, we'll subset the rows that include that provider...

In [ ]:

df.query("provider == 'iNaturalist.org'")

In [ ]:

df.dtypes

Exercise:¶

Extract the first 500 red wolf ("Canis rufus") records from the BISON API.
Can you create a table listing the records collected by the University of Kansas Biodiversity Institute?
Challenge: Can you create a table listing all the records collected in North Carolina?