#!/usr/bin/env python
# coding: utf-8

# ## Note: Sorry this notebook isn't working right now and is under development.
# 
# # Berkeley Air Quality Notebook

# **Welcome to our notebook on Berkeley Air Quality!** 
# 
# In this notebook we will be looking at Air Quality Index (AQI) scores in the surrounding Berkeley, CA area. With so many pollutants in the air, especially as we head into the annual fire season, AQI becomes something we check on daily. For many of us, this AQI map is all too familiar. Throughout this module we will discuss how data can be used to visualize and uncover underlying trends in the world.
# 
# **Let's get started!**

# <p align="center">
#   <img src="images/AQImap.webp" width="" height="" align="center">
# </p>

# <br>
# 
# ## Introduction to Jupyter Notebook

# Before we get started with the data, let's talk about what Jupyter Notebook is. This lab is set up in a Jupyter Notebook. Notebooks can contain anything from live code, to written text, equations or visualizations. The content of notebooks are written into rectangular sections called **cells**. 

# #### Types of Cells
# There are two types of cells in Jupyter, **code** cells and **markdown** cells. **Code cells**, as you can imagine, contain code in Python, the programming language that we will be using throughout this notebook.  **Markdown cells**, such as this one, contain written text. You can select any cell by clicking on it one. 

# #### Running Cells
# 'Running' a cell is similar to pressing 'Enter' on a calculator once you've typed in an expression; it computes all of the expressions contained within the cell.
# 
# To run a cell, you can do one of the following:
# 
# - press **Shift + Enter**
# - click the **Run** button on the top tool bar
# 
# Running a markdown cell will embed the text into the notebook and running a code cell will evaluate the code and display its output under the cell. 
# 
# Let's try it! **Run the code cell below.**

# In[1]:


print("Hello World!")


# #### Editing and Saving
# 
# - To **edit** a cell, simply double click on the desired cell and begin typing. The cell that you are currently working in will be highlighted by a green box.
# - To **save** the notebook, either click *Ctl + S* or navigate to the "File" dropdown and select "Save and Checkpoint"

# #### Adding Cells
# You can add a cell by clicking <b><code>Insert > Insert Cell Below</code></b> and choose the cell type in the drop down menu. Try adding a cell below to type in your name!
# 

# #### Deleting Cells 
# To delete a cell, click on the <b><code>scissors</code></b> at the top or <b><code>Edit > Cut Cells</code></b>. Delete the cell below.

# In[2]:


print("Delete this!")


# **Important Tip**: Everytime you open a Jupyter notebook, it is extremely important to run all the cells from the beginning in order for the notebook to work. 

# Now that we have had a brief crash course on Jupyter Notebooks, let's dive into Berkeley AQI!

# <br>
# 
# ## Introduction to the Data

# In this notebook we will look at data collected from PrupleAir, a company that manages a network of air quality sensors. The data from these sensors are then collected to create maps like the one displayed above that depicts an intuitive visualization of the air quality in a specific region. In the dataframe below, you will find several metrics that help us do this.
# 
# **Before we begin:**
# 
# - Click on <b><code>Cell</code></b> in the top toolbar  
# - Click on <b><code>Run All</code></b> in the drop down
# - Scroll back up to begin going through the notebook!

# In[3]:


import matplotlib.pyplot as plt
import numpy as np
import purpleair
import folium
import ipywidgets as widgets
from ipywidgets import interact, interactive, fixed, interact_manual
from datetime import datetime
from IPython.display import clear_output


# <br>
# 
# # PurpleAir Data

# Before we begin looking at data collected from PurpleAir sensors, lets first take a look at what a sensor is, and what it measures. 
# 
# 
# > Below is a picture of a real PurpleAir Air Quality Sensor. These sensor can be mounted both indoors or outdoors, and it tracks airborne particulate matter(PM) in real time using PMSX003 laser counters. Particulate matter can include things like dust, smoke, dirt and any other organic or inorganic particles in the air. With multiple sensors mounted in a region, PurpleAir can create a relatively accurate measure of AQI throughout the day as the air quality changes. 
# 
# For more information on how sensors work, take a look at the official PurpleAir website [here](https://www2.purpleair.com/community/faq#hc-what-do-the-numbers-on-the-purpleair-map-mean-1)!

# <p align="center">
#   <img src="images/purpleair-sensor-pm2.5.webp" width="" height="" align="center">
# </p>

# In order to work with the data, we need to pull it into our workspace. Fortunately, PurpleAir has created an API that allows users to pull in and work with their AQI data. In the code cell below we will import the purpleair API and use it to create a dataframe of data from all PurpleAir sensors, which is roughly ~20,000!
# 
# **Run the code cell below!**

# In[4]:


from purpleair.network import SensorList
p = SensorList()
df = p.to_dataframe(sensor_filter='all',
                    channel='parent')


# The dataframe below contains all the sensor data as of the latest update. It contains data on everything from the geograohical latitude and longitude of the sensor to data on the last time that sensor measured airborne PM.

# In[5]:


# Displaying dataframe with all the PurpleAir Sensor data
df


# Here is a breakdown of the dataframe above and what each column represents. 
# 
# 
# |Column Name   | Description |
# |--------------|---------|
# |lat |The latitude coordinate of the location  |
# |lon | The longitude coordinate of the location |
# |name |  The name of the location|
# |location_type | The nature of the location (ie. inside or outside) |
# |pm_2.5 | The level of fine particulate matter in the air of that location |
# |temp_f | The temperature of the location in degrees Farenheit|
# |temp_c | The temperature of the location in degrees Celsius|
# |humidity | The humidity percentage of the location|
# |pressure | The pressure index of the location (in millibars)|
# |last_seen | The last seen date and timestamp in UTC |
# |model |  Model of the specific sensor |
# |flagged | Whether or not the channel was marked as flagged (usually based on a fault)|
# |age | Sensor data age (when data was last received) |
# |10min_avg | Average PM 2.5 AQI over the last 10 minutes |
# |30min_avg | Average PM 2.5 AQI over the last 30 minutes |
# |1hour_avg | Average PM 2.5 AQI over the last hour|
# |6hour_avg | Average PM 2.5 AQI over the last 6 hours|
# |1day_avg | Average PM 2.5 AQI over the last day |
# |1week_avg |  Average PM 2.5 AQI over the last week|

# <br>
# 
# ### Airborne Particulate Matter (PM) 2.5 
# While many of the column names are relatively straightforward, such as the "name" column (which displays the set name of the particular sensor), the "location_type" column (which indicates whether it is an indoor or outdoor sensor), etc., we would like to draw your attention to the "pm_2.5" column. 
# 
# >The "pm_2.5" column represents the count of airborne pm that is larger than 2.5um/dl, in otherwords, airborne particles that have a diameter of 2.5 micrometers or less. In high levels, PM 2.5 particles can reduce visibility and cause the air to appear hazy. Tracking PM 2.5 is important because prolonged exposure to high levels of PM 2.5 particles can cause adverse US Environmental Protection Agency (EPA) use to calculate the local Air Quality Index (AQI).

# **QUESTION: Which item or object is closest to 1 micrometer?**
# 
# a) The length of an ant
# 
# b) The diameter of a spider web
# 
# c) The length of a grain of rice

# **ANSWER**
# 
# a) The length of an ant is typically 1 millimeter, which is 1,000 micrometers
# 
# **b) The diameter of a spider web is typically between 8 to 10 micrometers**
# 
# c) The length of a grain of rice is typically 6 millimeters which is 6,000 micrometers.

# If you go to the PurpleAir website [here](https://map.purpleair.com/1/mAQI/a10/p604800/cC0#14.67/37.87206/-122.26187), it should navigate you to a map of the surrounding Berkeley area. If you click on the some of the sensored located on UC Berkeley campus, you'll find that one of them is named "Le Conte Hall". 
# 
# Let's take a closer look at the Le Conte Hall Sensor! In the dataframe below we filter the dataframe by the sensor name ("Le Conte Hall") to pick out the row that corresponds to the specific sensor we are looking for. 

# In[6]:


df[df['name'] == "Le Conte Hall"]


# <br>
# 
# The row above gives us loads of information on the state of the AQI in Le Conte Hall at the present moment, but it would be nice to see the AQI information over time. Below is a dataframe that contains information about the Le Conte Hall sensor roughly over the last 7 days. We can do this by filtering the times each entry was created at.

# In[7]:


## data from Le Conte Hall sensor from the past week
from purpleair.sensor import Sensor
se = Sensor(77905)
le_conte = se.parent.get_historical(weeks_to_get=1,thingspeak_field='secondary')
le_conte['Date'] = [i.date().strftime("%d-%b-%Y") for i in le_conte['created_at']]
le_conte


# As you can see from the "created_at" column, the AQI was taken every two minutes over the past 7 days. The data frame also contains information on PM paticules of different diameters such as 0.3, 0.5, 1.0, 2.5, 5.0 and 10.0.
# 
# <br>
# 
# While this dataframe is useful, there are too many rows of data (~5000) to look at! Below is a widget that plots a line graph of the PM 2.5 measure over a specific day. 
# 
# **The drop down bar allows you to pick which day you would like graphed, so go ahead and pick a day!**

# In[8]:


def f(date):
    fig = plt.figure(figsize=(13,3))
    plt.plot(le_conte['created_at'].loc[le_conte['Date'] == date], le_conte["2.5um/dl"].loc[le_conte['Date'] == date])
    plt.xlabel('Time')
    plt.ylabel('PM 2.5 Particle Count')
    plt.title('Le Conte Hall Sensor PM 2.5')
    plt.rcParams["figure.figsize"] = (20,3)
    
interact(f, date = list(le_conte['Date'].unique()));


# The line plots above displays the date and hour along the x-axis and the PM 2.5 Particle count along the y-axis.
# 
# <br>

# **QUESTION: What is the highest index reading on the first time series plot?**

# *Your answer here*

# <br>
# 
# **QUESTION: What trends do you notice about the line plot?**

# *Your answer here*

# <br>
# 
# **QUESTION: Why do you think the index readings fluctuate from point to point?**

# *Your answer here*

# <br>
# 
# While the line plots do show us a trend in the PM2.5 count over time, we still have not clue how that translates to the API Index. The next section will discuss what AQI is and how it is calculated.

# ### API Index
# The API Index contains 6 categories that air quality can fall into. Each category contains a range of index values from 0 - 500 that is calculated from the regions PM 2.5 measure. The chart below is provided by the US Environmental Protection Agency (EPA) and shows the official AQI Index (these breakpoints were revised in 2012). 
# 
# For more information on how AQI Index is calculated, take a look at the AQI Index Factsheet provided by the EPA [here](https://www.epa.gov/sites/default/files/2016-04/documents/2012_aqi_factsheet.pdf)!

# <p align="center">
#   <img src="images/AQI-category.png" width="" height="" align="center">
# </p>

# <br>
# 
# **QUESTION: What is the difference between the original and revised breakpoints?**

# *Your answer here*

# <br>
# 
# **QUESTION: At 3:00 on November 30th, 2021 the PM 2.5 reading is 12.5. What category does it fall into?**
# 
# a) Good
# 
# b) Moderate
# 
# c) Unhealthy for Sensitive Groups

# **ANSWER: The category is Moderate because it falls into the 12.1 - 35.4 range.**

# Now that we know how sesors work, what they measure and how AQI Indexes are calculated, let's see if we can create a visualization of AQI Indexes that are a little closer to home!
# 
# First, let's find a group of sensors that are near UC Berkeley. The code cell below does just that. We use a range of longitude and latitude coordinates to decide whether to include or exclude a sensor. 

# In[9]:


## UC Berkeley,CA - Lat: 37.871666 / Lon: -122.272781

berkeleyData = df.loc[(df["lat"] >= 37.8) & (df["lat"] <= 37.9) & (df["lon"] >= -122.3) & (df["lon"] <= -122.2)]
berkeleyData = berkeleyData[["lat", "lon", "name", "location_type", "pm_2.5", "temp_f", "humidity", "pressure"]]
berkeleyData


# Now that we have a smaller subset of data to work with, the next step is to use the PM 2.5 measures to assign each sensor to an AQI Index Category and corresponding color. 

# In[11]:


#creating a column that indicates the AQI code name
color_code = []
for i in berkeleyData["pm_2.5"].to_list():
    if i <= 12.0:
        color_code.append('green')
    elif (i < 12) & (i <=35.4):
        color_code.append('yellow')
    elif (i < 35.5) & (i <=55.4):
        color_code.append('orange')   
    elif (i < 55.5) & (i <=150.4):
        color_code.append('red')
    elif (i < 150.5) & (i <=250.4):
        color_code.append('purple')
    else:
        color_code.append('darkpurple')


berkeleyData['code'] = color_code


# <br>
# 
# Our last step is to use the longitude and latitude coordinates to map the relative location of the sensor with is corresponding AQI Index color! The widget below contains two sliders. One represents the Latitude value and the other is the Longitude value. 
# 
# **Slide the sliders left and right to display a mapping of the sensors in that latitude and longitude region, or use your cursor to drag the mapping area.**
# 
# **Hint: Berkeley, CA - Lat: 37.871666 / Lon: -122.272781** 

# In[12]:


def map(Latitude ,Longitude):
    m = folium.Map(width=500, height=400, location=[Latitude, Longitude])
    
    for i in np.arange(len(berkeleyData) - 1):
        folium.Marker(
            location=[berkeleyData.iloc[i]['lat'], berkeleyData.iloc[i]['lon']],
            popup=berkeleyData.iloc[i]['name'],
            icon=folium.Icon(color=berkeleyData.iloc[i]['code']),
        ).add_to(m)
    display(m)
    
interact(map, Latitude = (36, 38, 0.001) , Longitude = (-123, -121, 0.001));
## UC Berkeley,CA - Lat: 37.871666 / Lon: -122.272781


# Now that we have created a map we can easily see what the AQI index is across the city! 

# <br>
# 
# **QUESTION: What do you notice about the map?**

# *Your answer here*

# <br>
# 
# Developed By: Melisa Esqueda, Maham Bawaney & Karalyn Chong