This analysis is attempting to identify factors associated with heavy traffic.
The dataset is comprised of daily and hourly entries for westbound traffic midway between Minneapolis and St. Paul, the largest and capital cities, respectively, in Minnesota. It was collected from a Department of Transportation station on interstate 94 between October 2012 and September 2018.
The dataset was created by John Hogue from the UCI Machine Learning Repository. https://archive.ics.uci.edu/ml/datasets/Metro+Interstate+Traffic+Volume
The colors used are the University of Minnesota school colors, gold for graphs and maroon for line plots. Go Gophers!
# import libraries, open file, set display options
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np
import pprint
pd.options.display.float_format = '{:20,.4f}'.format
traffic = pd.read_csv("Metro_Interstate_Traffic_Volume.csv")
pd.set_option('display.max_rows', None)
pd.set_option('display.max_columns', None)
pd.set_option('display.width', 1000)
pd.set_option('display.colheader_justify', 'center')
pd.set_option('display.precision', 3)
A quick look at the beginning and ending of the data set shows nine columns with a mix of categorical text and numeric information. The source of the data set states that an automatic traffic recorder (ATR) was used to tabulate traffic volume. It is expressed in the number of vehicles per hour.
# display head, tail, and basic info
display(traffic.head())
display(traffic.tail())
display(traffic.info())
display(traffic.describe())
# separate rain column by no rain or some rain
no_rain = traffic[traffic["rain_1h"] == 0]
some_rain = traffic[traffic["rain_1h"] > 0]
print("Hours with no rain: ", len(no_rain))
print("Hours with some rain", len(some_rain))
# separate snow column by no snow or some snow
no_snow = traffic[traffic["snow_1h"] == 0]
some_snow = traffic[traffic["snow_1h"] > 0]
print("Hours with no snow: ", len(no_snow))
print("Hours with some snow: ", len(some_snow))
# create a some_rain column with any rain measurements greater than 0
traffic["some_rain"] = traffic["rain_1h"][traffic["rain_1h"] > 0]
# basic stats for some_rain
display(traffic["some_rain"].describe())
# remove row with outlier rain values
traffic = traffic[traffic["rain_1h"]<100]
# basic stats for some_rain
display(traffic["some_rain"].describe())
# create a some_snow column with any snow measurements greater than 0
traffic["some_snow"] = traffic["snow_1h"][traffic["snow_1h"] > 0]
# basic stats for some_snow
display(traffic["some_snow"].describe())
# convert temperature column from Kelvin to Fahrenheit
def k_to_f(k_temp):
f_temp = ((k_temp-273.15)*(9/5)) + 32
return f_temp
traffic["temp"] = traffic["temp"].map(k_to_f)
# basic stats for temp column
display(traffic["temp"].describe())
# # look at outlier temp rows
# cold_days = traffic[traffic["temp"]<-100]
# display(cold_days)
# remove rows with outlier temperatures
traffic = traffic[traffic["temp"]>-100]
# basic stats for temp column
display(traffic["temp"].describe())
The previous statistics for traffic volume shows that the minimum value is 0, the maximum value is 7280, and that traffic is usually somewhere in the middle. A histogram shows an interesting distribution. The mean is heavily influenced by how often the traffic is really light and how often the traffic is fairly bad.
# create histogram of traffic volume for entire data set
traffic["traffic_volume"].plot.hist(color="#FFCC33")
plt.title("Vehicles per Hour")
plt.xticks([1000,3000,5000,7000], [1000,3000,5000,7000])
plt.yticks([2000,4000,6000,8000], [2000,4000,6000,8000])
plt.show()
Creating a daytime group (7:00 am to 6:00 pm) and a nighttime group (7:00 pm to 6:00 am) reveals unsurprising differences in traffic volume.
# transform column to datetime
traffic["date_time"] = pd.to_datetime(traffic["date_time"])
# create date_time series
day_night = traffic["date_time"].dt.hour
# create day traffic series from 0700 to 1800
day_traffic = day_night.between(7, 18)
day_traffic = traffic.loc[day_traffic].copy()
# create night traffic series from 1900 to 0600
night_traffic = day_night.between(19, 24) | day_night.between(0, 6)
night_traffic = traffic.loc[night_traffic].copy()
# verify
print("The number of rows and columns for the day and night series.")
display(day_traffic.shape)
display(night_traffic.shape)
# create two graphs
plt.figure(figsize=(10,7))
# daytime traffic volume histogram
plt.subplot(2,2,1)
day_traffic["traffic_volume"].plot.hist(color="#FFCC33")
plt.title("Daytime Vehicles per Hour")
plt.xticks([1000,3000,5000,7000], [1000,3000,5000,7000])
plt.xlim(0,7500)
plt.ylim(0,8000)
plt.ylabel("")
# nighttime traffic volume histogram
plt.subplot(2,2,2)
night_traffic["traffic_volume"].plot.hist(color="#FFCC33")
plt.title("Nighttime Vehicles per Hour")
plt.xticks([1000,3000,5000,7000], [1000,3000,5000,7000])
plt.xlim(0,7500)
plt.ylim(0,8000)
plt.ylabel("")
plt.show()
# basic daytime and nighttime stats
display(day_traffic.describe())
display(night_traffic.describe())
# create month column and convert info in date_time column to months 1-12
day_traffic["month"] = day_traffic["date_time"].dt.month
# group daytime by month and get the mean column values for each month
day_traffic_month_group = day_traffic.groupby("month").mean()
# basic stats for daytime month group mean traffic volume
display(day_traffic_month_group["traffic_volume"].describe())
# line plot of daytime month group mean traffic volumes
day_traffic_month_group["traffic_volume"].plot.line(c="#7A0019")
plt.title("Daytime Mean Traffic Volume by Month")
plt.yticks([4400,4600,4800], [4400,4600,4800])
plt.xlabel("")
plt.xticks([1,2,3,4,5,6,7,8,9,10,11,12],["Jan","Feb","Mar","Apr","May","Jun",
"Jul","Aug","Sep","Oct","Nov","Dec"])
plt.show()
# group July by year
july_months = traffic[traffic["date_time"].dt.month == 7]
july_yearly_group = july_months.groupby(traffic["date_time"].dt.year)
# line plot of July yearly group mean traffic volume
july_yearly_group["traffic_volume"].mean().plot.line(c="#7A0019")
plt.title("July Mean Traffic Volume by Year")
plt.xlabel("")
plt.show()
# create day_of_week column and convert info in date_time column to days 0-6
day_traffic["day_of_week"] = day_traffic["date_time"].dt.dayofweek
# group daytime by day of week and get the mean column values for each day
day_traffic_day_group = day_traffic.groupby("day_of_week").mean()
# line plot of daytime week of day group mean traffic volumes
day_traffic_day_group["traffic_volume"].plot.line(c="#7A0019")
plt.title("Daytime Mean Traffic Volume by Day")
plt.yticks([3500,4000,4500,5000],[3500,4000,4500,5000])
plt.xlabel("")
plt.xticks([0,1,2,3,4,5,6],["Mon","Tue","Wed","Thur","Fri","Sat","Sun"])
plt.show()
# create an hour column and convert info in date_time column to hours 0 to 23
day_traffic["hour"] = day_traffic["date_time"].dt.hour
# group daytime into weekdays (4=Friday) and weekends (5=Saturday)
weekdays = day_traffic[day_traffic["day_of_week"] <= 4]
weekends = day_traffic[day_traffic["day_of_week"] >= 5]
# group weekday entries by hour and get the mean column values for each hour
weekdays_hours_group = weekdays.groupby("hour").mean()
# group weekend entries by hour and get the mean column values for each hour
weekends_hours_group = weekends.groupby("hour").mean()
# basic stats for daytime weekday hour group mean traffic volume
# display(weekdays_hours_group["traffic_volume"].describe())
# create two graphs
plt.figure(figsize=(10,6))
# line plot of weekdays hours group mean traffic volumes
plt.subplot(2,2,1)
weekdays_hours_group["traffic_volume"].plot.line(c="#7A0019")
plt.title("Weekday Traffic Volume by Hour")
plt.ylim(0,6500)
# line plot of weekends hours group mean traffic volumes
plt.subplot(2,2,2)
weekends_hours_group["traffic_volume"].plot.line(c="#7A0019")
plt.title("Weekend Traffic Volulme by Hour")
plt.ylim(0,6500)
plt.show()
# line plot for nighttime traffic vollume by hour
night_traffic["hour"] = night_traffic["date_time"].dt.hour
nighttime_hours_group = night_traffic.groupby("hour").mean()
nighttime_hours_group["traffic_volume"].plot.bar(color="#FFCC33")
plt.ylim(0,6000)
plt.title("Nightime Mean Traffic Volume by Hour")
plt.xlabel("")
plt.show()
There are small correlations between daytime traffic volume and measurable amounts of snow and rain. Neither of these associations would be obvious though.
# Pearson correlation coefficient for all daytime columns
day_traffic.corr()
# scatter plot for daytime traffic volume and some snow
plt.scatter(day_traffic["traffic_volume"],day_traffic["some_snow"],c="#FFCC33")
plt.title("Traffic Volume and Snow")
plt.xlabel("Vehicles per Hour")
plt.ylabel("Snow in mm per Hour")
plt.show()
# scatter plot for daytime traffic volume and some rain
plt.scatter(day_traffic["traffic_volume"], day_traffic["some_rain"],c="#FFCC33")
plt.title("Traffic and Rain")
plt.xlabel("Vehicles per Hour")
plt.ylabel("Rain in mm per Hour")
plt.show()
weather_main
column provides an interesting graph but no real insight. The traffic volume seems to be fairly similar across the group.weather_description
doesn't reveal any surprises either. The highest traffic volumes were on days with snow, so maybe we can conclude the Minnesotans are pretty resilient to bad weather!# group day traffic by weather main and weather description
weather_main = day_traffic.groupby("weather_main").mean()
weather_description = day_traffic.groupby("weather_description").mean()
# bar chart for weather main
weather_main["traffic_volume"].plot.barh(color="#FFCC33")
plt.xticks([1000,3000,5000], [1000,3000,5000])
plt.ylabel("")
plt.title("Weather and Daytime Traffic Volume")
plt.show()
# bar chart for weather description
weather_description["traffic_volume"].plot.barh(figsize=(7,10), color="#FFCC33")
plt.xticks([1000,3000,5000], [1000,3000,5000])
plt.ylabel("")
plt.title("Weather and Daytime Traffic Volume")
plt.show()