Visualizing Earnings Based On College Majors¶
Dataset Description : We'll be working with a dataset on the job outcomes of students who graduated from college between 2010 and 2012. The original data on job outcomes was released by American Community Survey, which conducts surveys and aggregates the data. FiveThirtyEight cleaned the dataset and released it on their Github repo.
Attribute description : Each row in the dataset represents a different major in college and contains information on gender diversity, employment rates, median salaries, and more. Here are some of the columns in the dataset:
| Row | Description |
|---|---|
Rank |
Rank by median earnings (the dataset is ordered by this column) |
Major_code |
Major code |
Major |
Major description. |
Major_category |
Category of major. |
Total |
Total number of people with major. |
Sample_size |
Sample size (unweighted) of full-time. |
Men |
Male graduates. |
Women |
Female graduates. |
ShareWomen |
Women as share of total. |
Employed |
Number employed. |
Median |
Median salary of full-time, year-round workers. |
Low_wage_jobs |
Number in low-wage service jobs. |
Full_time |
Number employed 35 hours or more. |
Part_time |
Number employed less than 35 hours. |
Aim :¶
- Using 'scatter plots' we aim to determine : Do students in more popular majors make more money?
- Using 'histograms' we aim to determine : How many majors are predominantly male or female?
- Using 'bar plots' we aim to determine : Which category of majors have the most students?
Dataset cleaning :¶
Let's clean the dataset before performing any analysis so that we can extract clean and precise inferences from our dataset inspection.
# importing the required libraries:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
# in order to display the plot inline
%matplotlib inline
# reading and importing the dataset into a onadas dataframe:
recent_grads = pd.read_csv('recent-grads.csv')
recent_grads.iloc[0]
# Let's see how the data is structured :
recent_grads.head(5)
# Let's get the summary of all the numeric columns in the dataset:
recent_grads.describe()
# Let's get some info about the datatypes too :
recent_grads.info()
We will drop all the rows with missing values. Matplotlib expects that columns of values we pass in have matching lengths and missing values will cause matplotlib to throw errors.
# let's find all the missing values in the dataset:
raw_data_count = recent_grads.shape[0]
recent_grads.dropna(inplace = True)
cleaned_data_count = recent_grads.shape[0]
print ("The number of rows containing empty values were : {}" .format(raw_data_count-cleaned_data_count))
We found that there was only one row containing empty values and that was dropped. Now that the dataset is cleaned let's try to find some relations betweeen the various attributes of our dataset.
1. Using scatterplots to find the correlation between dataset attributes:¶
# 1. between 'sample_size' and 'employed':
recent_grads.plot(x='Sample_size' ,y= 'Employed',
kind = 'scatter',
title = 'Employed vs. Sample_size',
xlim = (0),
ylim = (0)
)
print("The correlation value between the attibutes are : {}".
format(recent_grads['Sample_size'].corr(recent_grads['Employed'])))
We see a strong positive correlation between the attributes.
# 2. between 'sample_size' and 'median':
recent_grads.plot(x= 'Sample_size', y= 'Median',
kind = 'scatter',
title = 'Sample_size vs. Median',
xlim = 0,
ylim = 0
)
print("The correlation value between the attibutes are : {}"
.format(recent_grads['Sample_size'].corr(recent_grads['Median'])))
Hence, we see a negative realtion between sample_size and median values in the dataset.
# 3. Between 'Full_time' and 'Median':
recent_grads.plot(x= 'Full_time', y= 'Median',
kind = 'scatter',
title = 'Full_time vs. Median',
xlim = 0,
ylim = 0
)
print("The correlation value between the attibutes are : {}".
format(recent_grads['Full_time'].corr(recent_grads['Median'])))
We see a negative realtion between Full_time and median values in the dataset.
# 3. Between 'ShareWomen' and 'Unemployment_rate':
recent_grads.plot(x= 'ShareWomen', y= 'Unemployment_rate',
kind = 'scatter',
title = 'ShareWomen vs. Unemployment_rate',
xlim = 0,
ylim = 0
)
print("The correlation value between the attibutes are : {}".
format(recent_grads['ShareWomen'].corr(recent_grads['Unemployment_rate'])))
We see a higly weak correlation between the values.
# 4. Between 'Men' and 'Median':
recent_grads.plot(x= 'Men', y= 'Median',
kind = 'scatter',
title = 'Men vs. Median',
xlim = 0,
ylim = 0
)
print("The correlation value between the attibutes are : {}".
format(recent_grads['Men'].corr(recent_grads['Median'])))
We again witness a very weak correlation between the given attributes.
# 5. Between 'Women' and 'Median':
recent_grads.plot(x= 'Women', y= 'Median',
kind = 'scatter',
title = 'Women vs. Median',
xlim = 0,
ylim = 0
)
print("The correlation value between the attibutes are : {}".
format(recent_grads['Women'].corr(recent_grads['Median'])))
We see a negative correlation between the attributes.
# 6.between 'sample_size' and 'Unemployment_rate':
recent_grads.plot(x= 'Sample_size', y= 'Unemployment_rate',
kind = 'scatter',
title = 'Sample_size vs. Unemployment_rate',
xlim = 0,
ylim = 0
)
print("The correlation value between the attibutes are : {}"
.format(recent_grads['Sample_size'].corr(recent_grads['Unemployment_rate'])))
We see a weak correlation between the attributes, similar to that between ShareWomen and Unemployment_rate.
# 7.between 'ShareWmen' and 'Median':
recent_grads.plot(x= 'ShareWomen', y= 'Median',
kind = 'scatter',
title = 'ShareWomen vs. Median',
xlim = 0,
ylim = 0
)
print("The correlation value between the attibutes are : {}"
.format(recent_grads['ShareWomen'].corr(recent_grads['Median'])))
We see a strong neegative correlation between 'ShareWomen' and'Median' attributes showing that : As the the percentage share of women increases, the median salary decreases.
2. Using histograms for futher exploration of the dataset attributes:¶
Let's generate histograms for columns in the dataset so as to explore their distributions.
# Let's make a list of all the columns we need to make plot the histogram for:
cols_hist = ['Sample_size', 'Median' , 'Employed', 'Full_time', 'ShareWomen', 'Unemployment_rate',
'Men', 'Women']
# Plotting the histograms using a loop:
# 1. 'Sample_size' :
recent_grads['Sample_size'].hist(bins= 20, range = (0, 5500) )
plt.title('Distribution of Sample_size')
plt.show()
# 2. 'Median' :
recent_grads['Median'].hist(bins= 20, xrot= 30)
plt.title('Distribution of Median')
plt.show()
# 2. 'Employed' :
recent_grads['Employed'].hist(bins= 20, xrot= 30, range = (0, 350000))
plt.title('Distribution of Employed')
plt.show()
# 3. 'Full_time' :
recent_grads['Full_time'].hist(bins= 20, xrot= 30)
plt.title('Distribution of Full_time')
plt.show()
# 4. 'ShareWomen'
recent_grads['ShareWomen'].hist(bins= 20, xrot= 30)
plt.title('Distribution of ShareWomen')
plt.show()
# 5. 'Unemployment_rate'
recent_grads['Unemployment_rate'].hist(bins= 25, xrot= 30)
plt.title('Distribution of Unemployment_rate')
plt.show()
# 6. 'Men'
recent_grads['Men'].hist(bins= 20, xrot= 30)
plt.title('Distribution of Men')
plt.show()
# 7. 'Women'
recent_grads['Women'].hist(bins= 30, xrot= 30)
plt.title('Distribution of Women')
plt.show()
3. Using scatter-matrix to find the correlation between dataset attributes:¶
A scatter matrix plot combines both scatter plots and histograms into one grid of plots and allows us to explore potential relationships and distributions simultaneously. A scatter matrix plot consists of n by n plots on a grid, where n is the number of columns, the plots on the diagonal are histograms, and the non-diagonal plots are scatter plots.
# 1. plotting 'Sample_size' and'Median' using scatter_matrix plots:
pd.plotting.scatter_matrix(recent_grads[['Sample_size', 'Median']], figsize=(10,10))
# 2.Let's plot a scatter matrix plot between sharewmen and low wage jobs:
pd.plotting.scatter_matrix(recent_grads[['ShareWomen', 'Low_wage_jobs', 'Unemployment_rate']], figsize= (10, 10))
# 3. creating a 3*3 scatter matrix plot:
pd.plotting.scatter_matrix(recent_grads[['Sample_size', 'Median', 'Unemployment_rate']], figsize= (10, 10))
# 4. creating a 3*3 scatter matrix plot:
pd.plotting.scatter_matrix(recent_grads[['ShareWomen', 'Median', 'Unemployment_rate']], figsize= (10, 10))
# a scatter matrix between 'ShareWomen' and'Employed':
pd.plotting.scatter_matrix(recent_grads[['ShareWomen', 'Employed']], figsize = (10, 10))
print ('The correlation value between the attributes are:\n {}' .format(recent_grads[['ShareWomen', 'Employed']].corr()))
4. Using bar plots to find the correlation between dataset attributes:¶
# Comparing the percentages of women in the first 10 and last 10 rows of our dataset:
recent_grads[:10]['ShareWomen'].plot(kind= 'bar')
plt.show()
print('The mean value of columns :{}' .format(recent_grads[:10]['ShareWomen'].mean()))
recent_grads[-10:]['ShareWomen'].plot(kind='bar')
plt.show()
print('The mean value of columns :{}' .format(recent_grads[-10:]['ShareWomen'].mean()))
We see that the percentage of women increases as we move down the rows of the dataset.
# Comparing the unemployment rate in the first 10 and last 10 rows of our dataset:
recent_grads[:10]['Unemployment_rate'].plot(kind= 'bar')
plt.show()
print('The mean value of columns :{}' .format(recent_grads[:10]['Unemployment_rate'].mean()))
recent_grads[-10:]['Unemployment_rate'].plot(kind='bar')
plt.show()
print('The mean value of columns :{}' .format(recent_grads[-10:]['Unemployment_rate'].mean()))
We see that the unemployment rate also increases as we move down the rows of the dataset.
Using the plots and values from above let's finally answer the questions that we sought to draw from the dataset:
Do students in more popular majors make more money? : Using the scatter plots, we see that for small
sample_sizethe range of median salary was higher, indicating that students in less popular majors make more money.Do students that majored in subjects that were majority female make more money? : Using the women vs median scatter plot we find subjects that had more females provide a lesser median salary.
Is there any link between the number of full-time employees and median salary?: From the scatter plot we found that lesser the number of full-time employees, higher is the range of the salary.
Let's if we can verify our findings numerically:
# Let's find the mean of the median salaries of the 10 most popular and 10 least popular majors:
top_tot = recent_grads['Total'].sort_values(ascending = False).head(10).values
med = 0;
for val in top_tot:
row = recent_grads[recent_grads['Total'] == val]
total_pop = row['Median']
med = med + int(total_pop)
print ('The mean of median salaries of the top 10 most popular majors is: {}'
.format(med/len(top_tot)))
low_tot = recent_grads['Total'].sort_values(ascending = False).tail(10).values
med = 0;
for val in low_tot:
row = recent_grads[recent_grads['Total'] == val]
total_pop = row['Median']
med = med + int(total_pop)
print ('The mean of median salaries of the 10 least popular majors is: {}'
.format(med/len(low_tot)))
Hence can conclusively prove that students from the lesser popular majors earn more than that of the popular ones.
# Let's find all the majors more populated with women and find the mean of their median salaries:
recent_grads[recent_grads['ShareWomen'] > 0.5]['Median'].mean()
# Let's compare our findings to the majors populated with men:
recent_grads[recent_grads['ShareWomen'] < 0.5]['Median'].mean()
We therefore see that majors that had more women, provided a lesser median salary on average.
# Let's find the mean of the median salaries of the 10 majors with most full_time employees to the least ones:
top_tot = recent_grads['Full_time'].sort_values(ascending = False).head(10).values
med = 0;
for val in top_tot:
row = recent_grads[recent_grads['Full_time'] == val]
total_pop = row['Median']
med = med + int(total_pop)
print ('The mean of median salaries of the majors with most full time emloyees: {}'
.format(med/len(top_tot)))
low_tot = recent_grads['Full_time'].sort_values(ascending = False).tail(10).values
med = 0;
for val in low_tot:
row = recent_grads[recent_grads['Full_time'] == val]
total_pop = row['Median']
med = med + int(total_pop)
print ('The mean of median salaries of the 10 majors with least full time employees: {}'
.format(med/len(low_tot)))
Hence we see that lesser the number of full_time employees, more the median salary on average.
Additional Exploration:¶
# grouped bar plot to compare the number of men with the number of women in each category of majors.
recent_grads.groupby('Major_category')['Men', 'Women'].agg('sum').plot.bar(figsize=(20, 7))
Observations :
- Most popular major category :
Business - Male dominated majors:
Engineering,Computers and Mathematics - Female dominated majors :
Arts,Biology & Life Sciences,Communications and Journalism,Education,Health,Humanities and Liberal Arts,Psychology and Social Work. - Equally populated majors :
Agriculture & Natural Resources,Business,Industrial Arts & Consumer Services,Law & Public Policy,Physical Sciences,Physical SciencesandSocial Science - Least popular major category :
Interdisciplinary(:combination of categories), otherwise :Communications & Journalism
# box plot to explore the distributions of median salaries and unemployment rate:
recent_grads['Median'].plot(kind='box', ylim=(0,80000)).grid(axis='y')
plt.show()
recent_grads['Unemployment_rate'].plot(kind='box').grid(axis='y')
Observations :
- Least median salary is in the lower 20's range.
- 50% of the salaries are between 30K-45K.
- 50% of the unemployment range is between 0.05-0.09 (i.e 5% - 9% of the total students enrolled in the majors.)
- 75% of the majors have unemployment rate below 9 percent
# Let's see if we can numerically verify our findings:
recent_grads[['Median', 'Unemployment_rate']].describe()
Conclusion:¶
- On average, less popular majors provide higher salaries.
- On average, the umemployment rate is 6.8%.
- Men have higher salaries than women, indicating a high income disparity.
- Branches like
Engineeringare still male dominant, whereas those likeHealthandEducationare female dominant.