Guided Project: Visualizing Earnings Based on College Majors


Introduction

We'll be working with a dataset on the job outcomes of students who graduated from college between 2010 and 2012. The original data on job outcomes was released by American Community Survey, which conducts surveys and aggregates the data. FiveThirtyEight cleaned the dataset and released it on their Github repo.

Each row in the dataset represents a different major in college and contains information on gender diversity, employment rates, median salaries, and more. Here are some of the columns in the dataset:

  • Rank - Rank by median earnings (the dataset is ordered by this column).
  • Major_code - Major code.
  • Major - Major description.
  • Major_category - Category of major.
  • Total - Total number of people with major.
  • Sample_size - Sample size (unweighted) of full-time.
  • Men - Male graduates.
  • Women - Female graduates.
  • ShareWomen - Women as share of total.
  • Employed - Number employed.
  • Median - Median salary of full-time, year-round workers.
  • Low_wage_jobs - Number in low-wage service jobs.
  • Full_time - Number employed 35 hours or more.
  • Part_time - Number employed less than 35 hours.

Next, we begin by exploring the data.

In [1]:
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
In [2]:
recent_grads = pd.read_csv('recent-grads.csv')
recent_grads.iloc[0]
Out[2]:
Rank                                        1
Major_code                               2419
Major                   PETROLEUM ENGINEERING
Total                                    2339
Men                                      2057
Women                                     282
Major_category                    Engineering
ShareWomen                           0.120564
Sample_size                                36
Employed                                 1976
Full_time                                1849
Part_time                                 270
Full_time_year_round                     1207
Unemployed                                 37
Unemployment_rate                   0.0183805
Median                                 110000
P25th                                   95000
P75th                                  125000
College_jobs                             1534
Non_college_jobs                          364
Low_wage_jobs                             193
Name: 0, dtype: object
In [3]:
recent_grads.head()
Out[3]:
Rank Major_code Major Total Men Women Major_category ShareWomen Sample_size Employed ... Part_time Full_time_year_round Unemployed Unemployment_rate Median P25th P75th College_jobs Non_college_jobs Low_wage_jobs
0 1 2419 PETROLEUM ENGINEERING 2339.0 2057.0 282.0 Engineering 0.120564 36 1976 ... 270 1207 37 0.018381 110000 95000 125000 1534 364 193
1 2 2416 MINING AND MINERAL ENGINEERING 756.0 679.0 77.0 Engineering 0.101852 7 640 ... 170 388 85 0.117241 75000 55000 90000 350 257 50
2 3 2415 METALLURGICAL ENGINEERING 856.0 725.0 131.0 Engineering 0.153037 3 648 ... 133 340 16 0.024096 73000 50000 105000 456 176 0
3 4 2417 NAVAL ARCHITECTURE AND MARINE ENGINEERING 1258.0 1123.0 135.0 Engineering 0.107313 16 758 ... 150 692 40 0.050125 70000 43000 80000 529 102 0
4 5 2405 CHEMICAL ENGINEERING 32260.0 21239.0 11021.0 Engineering 0.341631 289 25694 ... 5180 16697 1672 0.061098 65000 50000 75000 18314 4440 972

5 rows × 21 columns

In [4]:
recent_grads.tail()
Out[4]:
Rank Major_code Major Total Men Women Major_category ShareWomen Sample_size Employed ... Part_time Full_time_year_round Unemployed Unemployment_rate Median P25th P75th College_jobs Non_college_jobs Low_wage_jobs
168 169 3609 ZOOLOGY 8409.0 3050.0 5359.0 Biology & Life Science 0.637293 47 6259 ... 2190 3602 304 0.046320 26000 20000 39000 2771 2947 743
169 170 5201 EDUCATIONAL PSYCHOLOGY 2854.0 522.0 2332.0 Psychology & Social Work 0.817099 7 2125 ... 572 1211 148 0.065112 25000 24000 34000 1488 615 82
170 171 5202 CLINICAL PSYCHOLOGY 2838.0 568.0 2270.0 Psychology & Social Work 0.799859 13 2101 ... 648 1293 368 0.149048 25000 25000 40000 986 870 622
171 172 5203 COUNSELING PSYCHOLOGY 4626.0 931.0 3695.0 Psychology & Social Work 0.798746 21 3777 ... 965 2738 214 0.053621 23400 19200 26000 2403 1245 308
172 173 3501 LIBRARY SCIENCE 1098.0 134.0 964.0 Education 0.877960 2 742 ... 237 410 87 0.104946 22000 20000 22000 288 338 192

5 rows × 21 columns

In [5]:
recent_grads.describe()
Out[5]:
Rank Major_code Total Men Women ShareWomen Sample_size Employed Full_time Part_time Full_time_year_round Unemployed Unemployment_rate Median P25th P75th College_jobs Non_college_jobs Low_wage_jobs
count 173.000000 173.000000 172.000000 172.000000 172.000000 172.000000 173.000000 173.000000 173.000000 173.000000 173.000000 173.000000 173.000000 173.000000 173.000000 173.000000 173.000000 173.000000 173.000000
mean 87.000000 3879.815029 39370.081395 16723.406977 22646.674419 0.522223 356.080925 31192.763006 26029.306358 8832.398844 19694.427746 2416.329480 0.068191 40151.445087 29501.445087 51494.219653 12322.635838 13284.497110 3859.017341
std 50.084928 1687.753140 63483.491009 28122.433474 41057.330740 0.231205 618.361022 50675.002241 42869.655092 14648.179473 33160.941514 4112.803148 0.030331 11470.181802 9166.005235 14906.279740 21299.868863 23789.655363 6944.998579
min 1.000000 1100.000000 124.000000 119.000000 0.000000 0.000000 2.000000 0.000000 111.000000 0.000000 111.000000 0.000000 0.000000 22000.000000 18500.000000 22000.000000 0.000000 0.000000 0.000000
25% 44.000000 2403.000000 4549.750000 2177.500000 1778.250000 0.336026 39.000000 3608.000000 3154.000000 1030.000000 2453.000000 304.000000 0.050306 33000.000000 24000.000000 42000.000000 1675.000000 1591.000000 340.000000
50% 87.000000 3608.000000 15104.000000 5434.000000 8386.500000 0.534024 130.000000 11797.000000 10048.000000 3299.000000 7413.000000 893.000000 0.067961 36000.000000 27000.000000 47000.000000 4390.000000 4595.000000 1231.000000
75% 130.000000 5503.000000 38909.750000 14631.000000 22553.750000 0.703299 338.000000 31433.000000 25147.000000 9948.000000 16891.000000 2393.000000 0.087557 45000.000000 33000.000000 60000.000000 14444.000000 11783.000000 3466.000000
max 173.000000 6403.000000 393735.000000 173809.000000 307087.000000 0.968954 4212.000000 307933.000000 251540.000000 115172.000000 199897.000000 28169.000000 0.177226 110000.000000 95000.000000 125000.000000 151643.000000 148395.000000 48207.000000
In [6]:
raw_data_count = recent_grads.shape[0]
print("Rows number:", raw_data_count)
Rows number: 173

Now, we'll remove those rows that contain null values.

In [7]:
recent_grads = recent_grads.dropna()
cleaned_data_count = recent_grads.shape[0]
print("We got {} rows after removing those with null records.".format(cleaned_data_count))
We got 172 rows after removing those with null records.

Pandas, Scatter Plots

We are going to generate scatter charts to explore some relationships between fields (columns).

In [8]:
recent_grads.plot(x='Sample_size', y='Median', kind='scatter', title='Annual Median Salary vs Sample Size, Full-time Job')
plt.xlim(-100, 4500)
plt.ylim(10000, 120000)
Out[8]:
(10000, 120000)
In [9]:
recent_grads.plot(x='Sample_size', y='Unemployment_rate', kind='scatter', title='Unemployment Rate vs Sample Size, Full-time Job')
plt.xlim(-100, 4500)
plt.ylim(-0.01, 0.2)
Out[9]:
(-0.01, 0.2)
In [10]:
recent_grads.plot(x='Full_time', y='Median', kind='scatter', title='Annual Median Salary vs Full-time Employees')
plt.xlim(-10000, 275000)
plt.ylim(10000, 120000)
Out[10]:
(10000, 120000)
In [11]:
recent_grads.plot(x='ShareWomen', y='Unemployment_rate', kind='scatter', title='Unemployment Rate vs Share of Women in the Total')
plt.xlim(-0.05, 1.1)
plt.ylim(-0.02, 0.2)
Out[11]:
(-0.02, 0.2)
In [12]:
recent_grads.plot(x='Men', y='Median', kind='scatter', title='Annual Median Salary vs Men Graduates')
plt.xlim(-10000, 200000)
plt.ylim(10000, 120000)
Out[12]:
(10000, 120000)
In [13]:
recent_grads.plot(x='Women', y='Median', kind='scatter', title='Annual Median Salary vs Women Graduates')
plt.xlim(-10000, 325000)
plt.ylim(10000, 120000)
Out[13]:
(10000, 120000)

QUESTIONS

  • In the first scatter plot (Annual Median Salary vs Sample Size, Full-time Job) we can see that some of the less popular careers (values of the field 'Sample_size' between 0 and 400) seem to pay higher salaries, ranging from \$30'000 to \$65'000 approximately.

Do students that majored in subjects that were majority female make more money?

  • In the last graph ('Annual Median Salary vs Women Graduates') it shows a slight negative correlation unlike the diagram of male graduates. We'll confirm this answer later.
  • In the graph 'Average annual salary vs. full-time employees' we observe a slight positive correlation between the number of full-time employees (in the range of \$3'000 or less) and average annual salaries which are in the range of \$25'000 to \$50'000 approximately.

All answers will be validated later.


Pandas, Histograms

We'll generate histograms to explore the distribution of some fields:

In [14]:
recent_grads['Sample_size'].hist(bins=10, range=(0, 5000))
plt.title("Distribution - Sample Size, Full Time Job")
plt.xlabel("Sample Size")
plt.ylabel("Frequency")
Out[14]:
<matplotlib.text.Text at 0x7f32d5d68cf8>
In [15]:
recent_grads['Median'].hist(bins=12, range=(0, 120000))
plt.title("Distribution - Annual Median Salary")
plt.xlabel("Median")
plt.ylabel("Frequency")
Out[15]:
<matplotlib.text.Text at 0x7f32d5c92f98>
In [16]:
recent_grads['Employed'].hist(bins=14, range=(0, 350000))
plt.title("Distribution - Number of Employees")
plt.xlabel("Employed")
plt.ylabel("Frequency")
plt.ylim(0, 130)
Out[16]:
(0, 130)
In [17]:
recent_grads['Full_time'].hist(bins=12, range=(0, 300000))
plt.title("Distribution - Full Time Employees")
plt.xlabel("Full_time")
plt.ylabel("Frequency")
Out[17]:
<matplotlib.text.Text at 0x7f32d5e1a550>
In [18]:
recent_grads['ShareWomen'].hist(bins=10, range=(0, 1))
plt.title("Distribution - Share of Women in the Total")
plt.xlabel("ShareWomen")
plt.ylabel("Frequency")
Out[18]:
<matplotlib.text.Text at 0x7f32d5c17fd0>
In [19]:
recent_grads['Unemployment_rate'].hist(bins=8, range=(0, 0.2))
plt.title("Distribution - Unemployment Rate")
plt.xlabel("Unemployment_rate")
plt.ylabel("Frequency")
Out[19]:
<matplotlib.text.Text at 0x7f32d5d45ef0>
In [20]:
recent_grads['Men'].hist(bins=9, range=(0, 180000))
plt.title("Distribution - Men Graduates")
plt.xticks(rotation=45)
plt.xlabel("Men")
plt.ylabel("Frequency")
Out[20]:
<matplotlib.text.Text at 0x7f32d5d96e48>
In [21]:
recent_grads['Women'].hist(bins=14, range=(0, 350000))
plt.title("Distribution - Women Graduates")
plt.xlabel("Women")
plt.ylabel("Frequency")
Out[21]:
<matplotlib.text.Text at 0x7f32d7faa3c8>

Remarks

  • The distribution plots for the fields: 'Total' (total majors), 'Sample_size' (full-time sample size employees), 'Employed' (number of employees), 'Full_time' (employees with 35 hrs per week or more), 'Men' (male graduates) and 'Women' (female graduates) show a positive asymmetric distribution. These distributions are expected to be based on the total number of graduating students among the main majors during the period from 2010 to 2012.
  • The 'Unemployment_rate' field (unemployment rate by specialty) shows an almost normal distribution whose highest frequency is just over 60%.
  • The distribution of the "SharedWomen" field (women proportion on total graduates) does not show a specific pattern.
  • Graduate students of the most popular majors have a mean annual salary between \$20k and \$60k, with the highest frequency being between \$30k and \$40k.

Pandas, Scatter Matrix Plot

This type of plot combines both scatter plots and histograms into one grid of plots and allows us to explore potential relationships and distributions simultaneously.

To use it, we first import the function called 'scatter_matrix ()' from the 'pandas.plotting' module and then we will develop some diagrams that will help us to validate the answers to the previous questionnaire.

In [22]:
from pandas.plotting import scatter_matrix
In [23]:
scatter_matrix(recent_grads[['Sample_size', 'Median']], figsize=(10, 10))
Out[23]:
array([[<matplotlib.axes._subplots.AxesSubplot object at 0x7f32d7f35668>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f32d5e13c18>],
       [<matplotlib.axes._subplots.AxesSubplot object at 0x7f32d5a91400>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f32d5a47a90>]],
      dtype=object)
In [24]:
scatter_matrix(recent_grads[['Sample_size', 'Median', 'Unemployment_rate']], figsize=(15, 15))
Out[24]:
array([[<matplotlib.axes._subplots.AxesSubplot object at 0x7f32d59ed908>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f32d5967438>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f32d58afda0>],
       [<matplotlib.axes._subplots.AxesSubplot object at 0x7f32d58eca58>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f32d5835b70>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f32d57fc128>],
       [<matplotlib.axes._subplots.AxesSubplot object at 0x7f32d57c4828>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f32d577cf28>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f32d574e2b0>]],
      dtype=object)
In [25]:
scatter_matrix(recent_grads[['Total', 'Median']], figsize=(10, 10))
Out[25]:
array([[<matplotlib.axes._subplots.AxesSubplot object at 0x7f32d580ee80>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f32d5608240>],
       [<matplotlib.axes._subplots.AxesSubplot object at 0x7f32d55d2588>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f32d5586f60>]],
      dtype=object)

The plots above confirm that popular majors earn a lower mean annual salary than less popular specialties.

*Do students that majored in subjects that were majority female make more money?

In [26]:
scatter_matrix(recent_grads[['ShareWomen', 'Median']], figsize=(10, 10))
Out[26]:
array([[<matplotlib.axes._subplots.AxesSubplot object at 0x7f32d55b9588>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f32d54a3a58>],
       [<matplotlib.axes._subplots.AxesSubplot object at 0x7f32d53f44e0>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f32d5429f28>]],
      dtype=object)

The diagram confirms what we had previously concluded: as the proportion of women in a specialty increases, there is a gradual fall in the average annual salary.

In [27]:
scatter_matrix(recent_grads[['Full_time', 'Median']], figsize=(10, 10))
Out[27]:
array([[<matplotlib.axes._subplots.AxesSubplot object at 0x7f32d545b588>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f32d52c8b00>],
       [<matplotlib.axes._subplots.AxesSubplot object at 0x7f32d5294438>,
        <matplotlib.axes._subplots.AxesSubplot object at 0x7f32d52520b8>]],
      dtype=object)

We can see that as the number of full-time employees increases, there is a clear decrease in the annual median salary.


Pandas, Bar Plots

In [28]:
recent_grads.head(10).plot.barh(x='Major', y='ShareWomen', grid=True, color='purple')
plt.title("Share of Women in the Total - First 10 rows")
Out[28]:
<matplotlib.text.Text at 0x7f32d588e780>
In [29]:
recent_grads.tail(10).plot.barh(x='Major', y='ShareWomen', grid=True, color='purple')
plt.title("Share of Women in the Total - Last 10 rows")
Out[29]:
<matplotlib.text.Text at 0x7f32d404c550>

Remarks

  • The proportion of women with respect to the total of graduates by specialty is lower in the diagram of the first 10 rows (first graph) whose values range between 10% and 50%. Unlike the diagram of the last 10 records, whose values range between 60% and 95%.
  • 'Astronomy and Astrophysics', 'Actuarial Science' and 'Chemical Engineering' majors have a higher proportion of female graduates in the diagram of the first 10 rows (first graph).
  • The majors 'Communication Disorders Sciences and Services', 'Early Childhood Education' and 'Library Science' have a higher proportion of female graduates in the diagram of the last 10 rows (second graph).
In [30]:
recent_grads.head(10).plot.barh(x='Major', y='Unemployment_rate', grid=True, color='crimson')
plt.title("Unemployment Rate - First 10 rows")
Out[30]:
<matplotlib.text.Text at 0x7f32cf7af2e8>
In [31]:
recent_grads.tail(10).plot.barh(x='Major', y='Unemployment_rate', grid=True, color='crimson')
plt.title("Unemployment Rate - Last 10 rows")
Out[31]:
<matplotlib.text.Text at 0x7f32cf6c8668>

Remarks

  • Unemployment rates are very similar in both groups. It is slightly lower in the group of the first 10 records.
  • The "Metallurgical Engineering", "Astronomy and Astrophysics" and "Petroleum Engineering" majors have the lowest unemployment rates (less than 3%) among both groups.
  • The 'Nuclear Engineering' and 'Clinical Psychology' majors show the highest unemployment rates, with almost 18% and 15% respectively among both groups of records.

Next Steps

We compare the number of male graduates with the number of female graduates in each major category.

In [32]:
recent_grads[['Major_category', 'Men', 'Women']].groupby('Major_category').sum().plot.barh(title='Total Graduated by Gender in each Major Category', figsize=(12, 15))
plt.xlabel("Number of graduated")
Out[32]:
<matplotlib.text.Text at 0x7f32cf736668>

Remarks

  • The number of female graduates is higher than the number of male graduates in just over three-quarters of all categories.
  • The categories of 'Education', 'Health' and 'Psychology & Social Work' show a higher proportion of women graduates.
  • Engineering is the category with the highest proportion of male graduates.
  • 'Business' is the most popular category with a very similar number of male and female graduates.
  • The 'Interdisciplinary' category shows the lowest number of graduates, with a predominance of female graduates.

Exploring the distributions of median salaries and the unemployment rate

In [33]:
recent_grads['Median'].plot(kind='box', ylim=(0, 80000)).grid(color='purple', axis='y')
plt.title("Boxplot - Annual Median Salary")
Out[33]:
<matplotlib.text.Text at 0x7f32cf559208>
  • We note that the mean value is approximately \$35k.
  • About 75% of median annual salaries range from \$32k to \$62k.
  • The lowest median annual salary is \$22k.
  • Finally, some outliers are observed above \$62k.
In [34]:
recent_grads['Unemployment_rate'].plot(kind='box', ylim=(0, 0.16)).grid(color='green', axis='y')
plt.title("Boxplot - Unemployment Rate")
Out[34]:
<matplotlib.text.Text at 0x7f32cf4d1080>
  • The average unemployment rate among the specialties is approximately 7%.
  • Seventy-five percent of the specialties have an unemployment rate of just under 9%.
  • Some outliers are observed greater than 13% approximately.
  • We appreciate that a quarter of graduates have an unemployment rate of just under 5%.

Density in Scatter Plots

In [35]:
fig, ax = plt.subplots(nrows=3, ncols=2, figsize=(10,12))
plt.subplots_adjust(hspace=.5)

recent_grads.plot.hexbin(x='Sample_size', y='Median', gridsize=30, ax=ax[0,0], rot=45)
recent_grads.plot.hexbin(x='Sample_size', y='Unemployment_rate', gridsize=30, ax=ax[0,1], rot=45)
recent_grads.plot.hexbin(x='Full_time', y='Median', gridsize=30, ax=ax[1,0], rot=45)
recent_grads.plot.hexbin(x='ShareWomen', y='Unemployment_rate', gridsize=30, ax=ax[1,1])
recent_grads.plot.hexbin(x='Men', y='Median', gridsize=30, ax=ax[2,0], rot=45)
recent_grads.plot.hexbin(x='Women', y='Median', gridsize=30, ax=ax[2,1], rot=45)
Out[35]:
<matplotlib.axes._subplots.AxesSubplot at 0x7f32cf36b9e8>

Conclusion

  • The most popular specialties show a lower annual median salary compared to the less popular specialties. This may be due to the large volume of graduates in the most popular specialties, which can lead to employers reducing wages.
  • Unfortunately, the results still show us income inequality between male and female graduates. The majors with the highest number of women tend to have the lowest incomes.
  • Engineering careers have a greater presence of the male gender.
  • Majors related to human sciences such as education, health, etc. show a higher proportion of female graduates compared to the number of male graduates.
  • In general, the unemployment rate is approximately 7%.