In an earlier session, we used urllib and pandas to retrieve data from a static URL. Here we refine that procedure to again retrieve water flow data from the NWIS website. What's different this time, however, is that we examine the URL as a web service, dissecting it into its components (web service address and web service parameters). We also switch to using the Python requests package to handle the interaction, as it's slightly more user friendly than the urllib package.
The URL listing current discharge conditions in the Eno near Durham gage site is here:
http://waterdata.usgs.gov/nwis/uv?cb_00060=on&cb_00065=on&format=rdb&site_no=02085070&period=1&begin_date=&end_date=
As you'll see in the example below, the requests package constructs a web service request with two parameters. The first is the full service address (http://waterdata.usgs.gov/nwis/uv), which is followed by the set of service parameters, passed as a Python dictionary.
#import packages
import requests
#dissect the URL into it's components, for easy comprehension -- and easy modification
serviceURL = 'http://waterdata.usgs.gov/nwis/uv'
parameters = {'cb_00060':'on',
'cb_00065':'on',
'format':'rdb',
'period':'1',
'site_no':'02085070',
'begin_date':'',
'end_date':''
}
Ok, here we send the request, storing the response in a variable called response. The second line extacts the response as raw text into the variable responseText.
#Pass the url and its parameters to the server and get its response
response = requests.get(serviceURL, parameters)
responseText = response.text
#What did we get?
print(responseText)
If you scroll all the way to the last line in the file, you get the most recent reading. We can use some Python to print out what this reading is...
#Convert the response text into a list of lines and print the second to last line
responseLines = responseText.split('\n')
lastRecord = responseLines[-2].split('\t')
site = lastRecord[1]
cfs = lastRecord[4]
date = lastRecord[2]
print("Site {0} recorded a discharge of {1} cfs on {2}".format(site,cfs,date))
► EXERCISE: The 3 code blocks below are copies of the ones above. Modify these code blocks to get data for site 02085039 (Eno River At Cole Mill Rd Nr Huckleberry Spring), and print out its most recent gage height (in feet):
#dissect the URL into it's components, for easy comprehension -- and easy modification
serviceURL = 'http://waterdata.usgs.gov/nwis/uv'
parameters = {'cb_00060':'on',
'cb_00065':'on',
'format':'rdb',
'period':'1',
'site_no':'02085070',
'begin_date':'',
'end_date':''
}
#Pass the url and its parameters to the server and get its response
response = requests.get(serviceURL, parameters)
responseText = response.text
#Convert the response text into a list of lines and print the second to last line
responseLines = responseText.split('\n')
lastRecord = responseLines[-2].split('\t')
site = lastRecord[1]
cfs = lastRecord[4]
date = lastRecord[2]
print("Site {0} recorded a discharge of {1} cfs on {2}".format(site,cfs,date))