Cleaning and Analysing Employee Exit Survey Data

The stakeholders of two institutes want to know more about why employees leave their organisations. Specifically, they want to know the following:

  • Are employees who only worked for the institutes for a short period of time resigning due to some kind of dissatisfaction? What about employees who have been there longer?
  • Are younger employees resigning due to some kind of dissatisfaction? What about older employees?

The data that has been provided contains information on employee exit from two different institutes. The stakeholders have asked for these results to be combined. The aim is therefore to combine the results for both surveys and answer the questions posed about. However, although both used the same survey template, one of them customized some of the answers so before the datasets can be combined, the data needs to be cleaned.

In [143]:
import pandas as pd
import numpy as np

Explore the datasets

In [2]:
dete_survey = pd.read_csv("dete_survey.csv")
tafe_survey = pd.read_csv("tafe_survey.csv")

print(dete_survey.info())
print(tafe_survey.info())
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 822 entries, 0 to 821
Data columns (total 56 columns):
ID                                     822 non-null int64
SeparationType                         822 non-null object
Cease Date                             822 non-null object
DETE Start Date                        822 non-null object
Role Start Date                        822 non-null object
Position                               817 non-null object
Classification                         455 non-null object
Region                                 822 non-null object
Business Unit                          126 non-null object
Employment Status                      817 non-null object
Career move to public sector           822 non-null bool
Career move to private sector          822 non-null bool
Interpersonal conflicts                822 non-null bool
Job dissatisfaction                    822 non-null bool
Dissatisfaction with the department    822 non-null bool
Physical work environment              822 non-null bool
Lack of recognition                    822 non-null bool
Lack of job security                   822 non-null bool
Work location                          822 non-null bool
Employment conditions                  822 non-null bool
Maternity/family                       822 non-null bool
Relocation                             822 non-null bool
Study/Travel                           822 non-null bool
Ill Health                             822 non-null bool
Traumatic incident                     822 non-null bool
Work life balance                      822 non-null bool
Workload                               822 non-null bool
None of the above                      822 non-null bool
Professional Development               808 non-null object
Opportunities for promotion            735 non-null object
Staff morale                           816 non-null object
Workplace issue                        788 non-null object
Physical environment                   817 non-null object
Worklife balance                       815 non-null object
Stress and pressure support            810 non-null object
Performance of supervisor              813 non-null object
Peer support                           812 non-null object
Initiative                             813 non-null object
Skills                                 811 non-null object
Coach                                  767 non-null object
Career Aspirations                     746 non-null object
Feedback                               792 non-null object
Further PD                             768 non-null object
Communication                          814 non-null object
My say                                 812 non-null object
Information                            816 non-null object
Kept informed                          813 non-null object
Wellness programs                      766 non-null object
Health & Safety                        793 non-null object
Gender                                 798 non-null object
Age                                    811 non-null object
Aboriginal                             16 non-null object
Torres Strait                          3 non-null object
South Sea                              7 non-null object
Disability                             23 non-null object
NESB                                   32 non-null object
dtypes: bool(18), int64(1), object(37)
memory usage: 258.6+ KB
None
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 702 entries, 0 to 701
Data columns (total 72 columns):
Record ID                                                                                                                                                        702 non-null float64
Institute                                                                                                                                                        702 non-null object
WorkArea                                                                                                                                                         702 non-null object
CESSATION YEAR                                                                                                                                                   695 non-null float64
Reason for ceasing employment                                                                                                                                    701 non-null object
Contributing Factors. Career Move - Public Sector                                                                                                                437 non-null object
Contributing Factors. Career Move - Private Sector                                                                                                               437 non-null object
Contributing Factors. Career Move - Self-employment                                                                                                              437 non-null object
Contributing Factors. Ill Health                                                                                                                                 437 non-null object
Contributing Factors. Maternity/Family                                                                                                                           437 non-null object
Contributing Factors. Dissatisfaction                                                                                                                            437 non-null object
Contributing Factors. Job Dissatisfaction                                                                                                                        437 non-null object
Contributing Factors. Interpersonal Conflict                                                                                                                     437 non-null object
Contributing Factors. Study                                                                                                                                      437 non-null object
Contributing Factors. Travel                                                                                                                                     437 non-null object
Contributing Factors. Other                                                                                                                                      437 non-null object
Contributing Factors. NONE                                                                                                                                       437 non-null object
Main Factor. Which of these was the main factor for leaving?                                                                                                     113 non-null object
InstituteViews. Topic:1. I feel the senior leadership had a clear vision and direction                                                                           608 non-null object
InstituteViews. Topic:2. I was given access to skills training to help me do my job better                                                                       613 non-null object
InstituteViews. Topic:3. I was given adequate opportunities for personal development                                                                             610 non-null object
InstituteViews. Topic:4. I was given adequate opportunities for promotion within %Institute]Q25LBL%                                                              608 non-null object
InstituteViews. Topic:5. I felt the salary for the job was right for the responsibilities I had                                                                  615 non-null object
InstituteViews. Topic:6. The organisation recognised when staff did good work                                                                                    607 non-null object
InstituteViews. Topic:7. Management was generally supportive of me                                                                                               614 non-null object
InstituteViews. Topic:8. Management was generally supportive of my team                                                                                          608 non-null object
InstituteViews. Topic:9. I was kept informed of the changes in the organisation which would affect me                                                            610 non-null object
InstituteViews. Topic:10. Staff morale was positive within the Institute                                                                                         602 non-null object
InstituteViews. Topic:11. If I had a workplace issue it was dealt with quickly                                                                                   601 non-null object
InstituteViews. Topic:12. If I had a workplace issue it was dealt with efficiently                                                                               597 non-null object
InstituteViews. Topic:13. If I had a workplace issue it was dealt with discreetly                                                                                601 non-null object
WorkUnitViews. Topic:14. I was satisfied with the quality of the management and supervision within my work unit                                                  609 non-null object
WorkUnitViews. Topic:15. I worked well with my colleagues                                                                                                        605 non-null object
WorkUnitViews. Topic:16. My job was challenging and interesting                                                                                                  607 non-null object
WorkUnitViews. Topic:17. I was encouraged to use my initiative in the course of my work                                                                          610 non-null object
WorkUnitViews. Topic:18. I had sufficient contact with other people in my job                                                                                    613 non-null object
WorkUnitViews. Topic:19. I was given adequate support and co-operation by my peers to enable me to do my job                                                     609 non-null object
WorkUnitViews. Topic:20. I was able to use the full range of my skills in my job                                                                                 609 non-null object
WorkUnitViews. Topic:21. I was able to use the full range of my abilities in my job. ; Category:Level of Agreement; Question:YOUR VIEWS ABOUT YOUR WORK UNIT]    608 non-null object
WorkUnitViews. Topic:22. I was able to use the full range of my knowledge in my job                                                                              608 non-null object
WorkUnitViews. Topic:23. My job provided sufficient variety                                                                                                      611 non-null object
WorkUnitViews. Topic:24. I was able to cope with the level of stress and pressure in my job                                                                      610 non-null object
WorkUnitViews. Topic:25. My job allowed me to balance the demands of work and family to my satisfaction                                                          611 non-null object
WorkUnitViews. Topic:26. My supervisor gave me adequate personal recognition and feedback on my performance                                                      606 non-null object
WorkUnitViews. Topic:27. My working environment was satisfactory e.g. sufficient space, good lighting, suitable seating and working area                         610 non-null object
WorkUnitViews. Topic:28. I was given the opportunity to mentor and coach others in order for me to pass on my skills and knowledge prior to my cessation date    609 non-null object
WorkUnitViews. Topic:29. There was adequate communication between staff in my unit                                                                               603 non-null object
WorkUnitViews. Topic:30. Staff morale was positive within my work unit                                                                                           606 non-null object
Induction. Did you undertake Workplace Induction?                                                                                                                619 non-null object
InductionInfo. Topic:Did you undertake a Corporate Induction?                                                                                                    432 non-null object
InductionInfo. Topic:Did you undertake a Institute Induction?                                                                                                    483 non-null object
InductionInfo. Topic: Did you undertake Team Induction?                                                                                                          440 non-null object
InductionInfo. Face to Face Topic:Did you undertake a Corporate Induction; Category:How it was conducted?                                                        555 non-null object
InductionInfo. On-line Topic:Did you undertake a Corporate Induction; Category:How it was conducted?                                                             555 non-null object
InductionInfo. Induction Manual Topic:Did you undertake a Corporate Induction?                                                                                   555 non-null object
InductionInfo. Face to Face Topic:Did you undertake a Institute Induction?                                                                                       530 non-null object
InductionInfo. On-line Topic:Did you undertake a Institute Induction?                                                                                            555 non-null object
InductionInfo. Induction Manual Topic:Did you undertake a Institute Induction?                                                                                   553 non-null object
InductionInfo. Face to Face Topic: Did you undertake Team Induction; Category?                                                                                   555 non-null object
InductionInfo. On-line Topic: Did you undertake Team Induction?process you undertook and how it was conducted.]                                                  555 non-null object
InductionInfo. Induction Manual Topic: Did you undertake Team Induction?                                                                                         555 non-null object
Workplace. Topic:Did you and your Manager develop a Performance and Professional Development Plan (PPDP)?                                                        608 non-null object
Workplace. Topic:Does your workplace promote a work culture free from all forms of unlawful discrimination?                                                      594 non-null object
Workplace. Topic:Does your workplace promote and practice the principles of employment equity?                                                                   587 non-null object
Workplace. Topic:Does your workplace value the diversity of its employees?                                                                                       586 non-null object
Workplace. Topic:Would you recommend the Institute as an employer to others?                                                                                     581 non-null object
Gender. What is your Gender?                                                                                                                                     596 non-null object
CurrentAge. Current Age                                                                                                                                          596 non-null object
Employment Type. Employment Type                                                                                                                                 596 non-null object
Classification. Classification                                                                                                                                   596 non-null object
LengthofServiceOverall. Overall Length of Service at Institute (in years)                                                                                        596 non-null object
LengthofServiceCurrent. Length of Service at current workplace (in years)                                                                                        596 non-null object
dtypes: float64(2), object(70)
memory usage: 395.0+ KB
None
In [3]:
dete_survey.head()
Out[3]:
ID SeparationType Cease Date DETE Start Date Role Start Date Position Classification Region Business Unit Employment Status ... Kept informed Wellness programs Health & Safety Gender Age Aboriginal Torres Strait South Sea Disability NESB
0 1 Ill Health Retirement 08/2012 1984 2004 Public Servant A01-A04 Central Office Corporate Strategy and Peformance Permanent Full-time ... N N N Male 56-60 NaN NaN NaN NaN Yes
1 2 Voluntary Early Retirement (VER) 08/2012 Not Stated Not Stated Public Servant AO5-AO7 Central Office Corporate Strategy and Peformance Permanent Full-time ... N N N Male 56-60 NaN NaN NaN NaN NaN
2 3 Voluntary Early Retirement (VER) 05/2012 2011 2011 Schools Officer NaN Central Office Education Queensland Permanent Full-time ... N N N Male 61 or older NaN NaN NaN NaN NaN
3 4 Resignation-Other reasons 05/2012 2005 2006 Teacher Primary Central Queensland NaN Permanent Full-time ... A N A Female 36-40 NaN NaN NaN NaN NaN
4 5 Age Retirement 05/2012 1970 1989 Head of Curriculum/Head of Special Education NaN South East NaN Permanent Full-time ... N A M Female 61 or older NaN NaN NaN NaN NaN

5 rows × 56 columns

In [4]:
tafe_survey.head()
Out[4]:
Record ID Institute WorkArea CESSATION YEAR Reason for ceasing employment Contributing Factors. Career Move - Public Sector Contributing Factors. Career Move - Private Sector Contributing Factors. Career Move - Self-employment Contributing Factors. Ill Health Contributing Factors. Maternity/Family ... Workplace. Topic:Does your workplace promote a work culture free from all forms of unlawful discrimination? Workplace. Topic:Does your workplace promote and practice the principles of employment equity? Workplace. Topic:Does your workplace value the diversity of its employees? Workplace. Topic:Would you recommend the Institute as an employer to others? Gender. What is your Gender? CurrentAge. Current Age Employment Type. Employment Type Classification. Classification LengthofServiceOverall. Overall Length of Service at Institute (in years) LengthofServiceCurrent. Length of Service at current workplace (in years)
0 6.341330e+17 Southern Queensland Institute of TAFE Non-Delivery (corporate) 2010.0 Contract Expired NaN NaN NaN NaN NaN ... Yes Yes Yes Yes Female 26 30 Temporary Full-time Administration (AO) 1-2 1-2
1 6.341337e+17 Mount Isa Institute of TAFE Non-Delivery (corporate) 2010.0 Retirement - - - - - ... Yes Yes Yes Yes NaN NaN NaN NaN NaN NaN
2 6.341388e+17 Mount Isa Institute of TAFE Delivery (teaching) 2010.0 Retirement - - - - - ... Yes Yes Yes Yes NaN NaN NaN NaN NaN NaN
3 6.341399e+17 Mount Isa Institute of TAFE Non-Delivery (corporate) 2010.0 Resignation - - - - - ... Yes Yes Yes Yes NaN NaN NaN NaN NaN NaN
4 6.341466e+17 Southern Queensland Institute of TAFE Delivery (teaching) 2010.0 Resignation - Career Move - Private Sector - - - ... Yes Yes Yes Yes Male 41 45 Permanent Full-time Teacher (including LVT) 3-4 3-4

5 rows × 72 columns

In [5]:
#count the number of NA values in each column
dete_survey.isnull().sum()
Out[5]:
ID                                       0
SeparationType                           0
Cease Date                               0
DETE Start Date                          0
Role Start Date                          0
Position                                 5
Classification                         367
Region                                   0
Business Unit                          696
Employment Status                        5
Career move to public sector             0
Career move to private sector            0
Interpersonal conflicts                  0
Job dissatisfaction                      0
Dissatisfaction with the department      0
Physical work environment                0
Lack of recognition                      0
Lack of job security                     0
Work location                            0
Employment conditions                    0
Maternity/family                         0
Relocation                               0
Study/Travel                             0
Ill Health                               0
Traumatic incident                       0
Work life balance                        0
Workload                                 0
None of the above                        0
Professional Development                14
Opportunities for promotion             87
Staff morale                             6
Workplace issue                         34
Physical environment                     5
Worklife balance                         7
Stress and pressure support             12
Performance of supervisor                9
Peer support                            10
Initiative                               9
Skills                                  11
Coach                                   55
Career Aspirations                      76
Feedback                                30
Further PD                              54
Communication                            8
My say                                  10
Information                              6
Kept informed                            9
Wellness programs                       56
Health & Safety                         29
Gender                                  24
Age                                     11
Aboriginal                             806
Torres Strait                          819
South Sea                              815
Disability                             799
NESB                                   790
dtype: int64

The dete_survey contains information about each employee such as age, gender, demographics and also questions about why they left the organisation. There's also lots of information stored here which may not be required for the analysis.

In [6]:
print(tafe_survey.shape)

tafe_survey.isnull().sum()
(702, 72)
Out[6]:
Record ID                                                                                                                                                          0
Institute                                                                                                                                                          0
WorkArea                                                                                                                                                           0
CESSATION YEAR                                                                                                                                                     7
Reason for ceasing employment                                                                                                                                      1
Contributing Factors. Career Move - Public Sector                                                                                                                265
Contributing Factors. Career Move - Private Sector                                                                                                               265
Contributing Factors. Career Move - Self-employment                                                                                                              265
Contributing Factors. Ill Health                                                                                                                                 265
Contributing Factors. Maternity/Family                                                                                                                           265
Contributing Factors. Dissatisfaction                                                                                                                            265
Contributing Factors. Job Dissatisfaction                                                                                                                        265
Contributing Factors. Interpersonal Conflict                                                                                                                     265
Contributing Factors. Study                                                                                                                                      265
Contributing Factors. Travel                                                                                                                                     265
Contributing Factors. Other                                                                                                                                      265
Contributing Factors. NONE                                                                                                                                       265
Main Factor. Which of these was the main factor for leaving?                                                                                                     589
InstituteViews. Topic:1. I feel the senior leadership had a clear vision and direction                                                                            94
InstituteViews. Topic:2. I was given access to skills training to help me do my job better                                                                        89
InstituteViews. Topic:3. I was given adequate opportunities for personal development                                                                              92
InstituteViews. Topic:4. I was given adequate opportunities for promotion within %Institute]Q25LBL%                                                               94
InstituteViews. Topic:5. I felt the salary for the job was right for the responsibilities I had                                                                   87
InstituteViews. Topic:6. The organisation recognised when staff did good work                                                                                     95
InstituteViews. Topic:7. Management was generally supportive of me                                                                                                88
InstituteViews. Topic:8. Management was generally supportive of my team                                                                                           94
InstituteViews. Topic:9. I was kept informed of the changes in the organisation which would affect me                                                             92
InstituteViews. Topic:10. Staff morale was positive within the Institute                                                                                         100
InstituteViews. Topic:11. If I had a workplace issue it was dealt with quickly                                                                                   101
InstituteViews. Topic:12. If I had a workplace issue it was dealt with efficiently                                                                               105
                                                                                                                                                                ... 
WorkUnitViews. Topic:25. My job allowed me to balance the demands of work and family to my satisfaction                                                           91
WorkUnitViews. Topic:26. My supervisor gave me adequate personal recognition and feedback on my performance                                                       96
WorkUnitViews. Topic:27. My working environment was satisfactory e.g. sufficient space, good lighting, suitable seating and working area                          92
WorkUnitViews. Topic:28. I was given the opportunity to mentor and coach others in order for me to pass on my skills and knowledge prior to my cessation date     93
WorkUnitViews. Topic:29. There was adequate communication between staff in my unit                                                                                99
WorkUnitViews. Topic:30. Staff morale was positive within my work unit                                                                                            96
Induction. Did you undertake Workplace Induction?                                                                                                                 83
InductionInfo. Topic:Did you undertake a Corporate Induction?                                                                                                    270
InductionInfo. Topic:Did you undertake a Institute Induction?                                                                                                    219
InductionInfo. Topic: Did you undertake Team Induction?                                                                                                          262
InductionInfo. Face to Face Topic:Did you undertake a Corporate Induction; Category:How it was conducted?                                                        147
InductionInfo. On-line Topic:Did you undertake a Corporate Induction; Category:How it was conducted?                                                             147
InductionInfo. Induction Manual Topic:Did you undertake a Corporate Induction?                                                                                   147
InductionInfo. Face to Face Topic:Did you undertake a Institute Induction?                                                                                       172
InductionInfo. On-line Topic:Did you undertake a Institute Induction?                                                                                            147
InductionInfo. Induction Manual Topic:Did you undertake a Institute Induction?                                                                                   149
InductionInfo. Face to Face Topic: Did you undertake Team Induction; Category?                                                                                   147
InductionInfo. On-line Topic: Did you undertake Team Induction?process you undertook and how it was conducted.]                                                  147
InductionInfo. Induction Manual Topic: Did you undertake Team Induction?                                                                                         147
Workplace. Topic:Did you and your Manager develop a Performance and Professional Development Plan (PPDP)?                                                         94
Workplace. Topic:Does your workplace promote a work culture free from all forms of unlawful discrimination?                                                      108
Workplace. Topic:Does your workplace promote and practice the principles of employment equity?                                                                   115
Workplace. Topic:Does your workplace value the diversity of its employees?                                                                                       116
Workplace. Topic:Would you recommend the Institute as an employer to others?                                                                                     121
Gender. What is your Gender?                                                                                                                                     106
CurrentAge. Current Age                                                                                                                                          106
Employment Type. Employment Type                                                                                                                                 106
Classification. Classification                                                                                                                                   106
LengthofServiceOverall. Overall Length of Service at Institute (in years)                                                                                        106
LengthofServiceCurrent. Length of Service at current workplace (in years)                                                                                        106
Length: 72, dtype: int64

In the tafe_survey dataframe there are 702 rows and 72 columns. Most of the columns have some missing data. This dataset contains similar information to the dete_survey however they are presented in a different form.

Clean dataset columns

In [7]:
#As there is lots of information in the dete survey that is not relevant, these columns will be dropped from the analysis. 

print(dete_survey.columns[28:49])
dete_survey = pd.read_csv("dete_survey.csv", na_values = "Not Stated")
dete_survey_updated = dete_survey.drop(dete_survey.columns[28:49], axis=1)
dete_survey_updated.shape
Index(['Professional Development', 'Opportunities for promotion',
       'Staff morale', 'Workplace issue', 'Physical environment',
       'Worklife balance', 'Stress and pressure support',
       'Performance of supervisor', 'Peer support', 'Initiative', 'Skills',
       'Coach', 'Career Aspirations', 'Feedback', 'Further PD',
       'Communication', 'My say', 'Information', 'Kept informed',
       'Wellness programs', 'Health & Safety'],
      dtype='object')
Out[7]:
(822, 35)
In [8]:
#similarly for the tafe survey, columns which are not relevant to this analysis will be dropped

print(tafe_survey.columns[17:66])
tafe_survey_updated = tafe_survey.drop(tafe_survey.columns[17:66], axis = 1)
tafe_survey_updated.shape

tafe_survey_updated.head()
Index(['Main Factor. Which of these was the main factor for leaving?',
       'InstituteViews. Topic:1. I feel the senior leadership had a clear vision and direction',
       'InstituteViews. Topic:2. I was given access to skills training to help me do my job better',
       'InstituteViews. Topic:3. I was given adequate opportunities for personal development',
       'InstituteViews. Topic:4. I was given adequate opportunities for promotion within %Institute]Q25LBL%',
       'InstituteViews. Topic:5. I felt the salary for the job was right for the responsibilities I had',
       'InstituteViews. Topic:6. The organisation recognised when staff did good work',
       'InstituteViews. Topic:7. Management was generally supportive of me',
       'InstituteViews. Topic:8. Management was generally supportive of my team',
       'InstituteViews. Topic:9. I was kept informed of the changes in the organisation which would affect me',
       'InstituteViews. Topic:10. Staff morale was positive within the Institute',
       'InstituteViews. Topic:11. If I had a workplace issue it was dealt with quickly',
       'InstituteViews. Topic:12. If I had a workplace issue it was dealt with efficiently',
       'InstituteViews. Topic:13. If I had a workplace issue it was dealt with discreetly',
       'WorkUnitViews. Topic:14. I was satisfied with the quality of the management and supervision within my work unit',
       'WorkUnitViews. Topic:15. I worked well with my colleagues',
       'WorkUnitViews. Topic:16. My job was challenging and interesting',
       'WorkUnitViews. Topic:17. I was encouraged to use my initiative in the course of my work',
       'WorkUnitViews. Topic:18. I had sufficient contact with other people in my job',
       'WorkUnitViews. Topic:19. I was given adequate support and co-operation by my peers to enable me to do my job',
       'WorkUnitViews. Topic:20. I was able to use the full range of my skills in my job',
       'WorkUnitViews. Topic:21. I was able to use the full range of my abilities in my job. ; Category:Level of Agreement; Question:YOUR VIEWS ABOUT YOUR WORK UNIT]',
       'WorkUnitViews. Topic:22. I was able to use the full range of my knowledge in my job',
       'WorkUnitViews. Topic:23. My job provided sufficient variety',
       'WorkUnitViews. Topic:24. I was able to cope with the level of stress and pressure in my job',
       'WorkUnitViews. Topic:25. My job allowed me to balance the demands of work and family to my satisfaction',
       'WorkUnitViews. Topic:26. My supervisor gave me adequate personal recognition and feedback on my performance',
       'WorkUnitViews. Topic:27. My working environment was satisfactory e.g. sufficient space, good lighting, suitable seating and working area',
       'WorkUnitViews. Topic:28. I was given the opportunity to mentor and coach others in order for me to pass on my skills and knowledge prior to my cessation date',
       'WorkUnitViews. Topic:29. There was adequate communication between staff in my unit',
       'WorkUnitViews. Topic:30. Staff morale was positive within my work unit',
       'Induction. Did you undertake Workplace Induction?',
       'InductionInfo. Topic:Did you undertake a Corporate Induction?',
       'InductionInfo. Topic:Did you undertake a Institute Induction?',
       'InductionInfo. Topic: Did you undertake Team Induction?',
       'InductionInfo. Face to Face Topic:Did you undertake a Corporate Induction; Category:How it was conducted?',
       'InductionInfo. On-line Topic:Did you undertake a Corporate Induction; Category:How it was conducted?',
       'InductionInfo. Induction Manual Topic:Did you undertake a Corporate Induction?',
       'InductionInfo. Face to Face Topic:Did you undertake a Institute Induction?',
       'InductionInfo. On-line Topic:Did you undertake a Institute Induction?',
       'InductionInfo. Induction Manual Topic:Did you undertake a Institute Induction?',
       'InductionInfo. Face to Face Topic: Did you undertake Team Induction; Category?',
       'InductionInfo. On-line Topic: Did you undertake Team Induction?process you undertook and how it was conducted.]',
       'InductionInfo. Induction Manual Topic: Did you undertake Team Induction?',
       'Workplace. Topic:Did you and your Manager develop a Performance and Professional Development Plan (PPDP)?',
       'Workplace. Topic:Does your workplace promote a work culture free from all forms of unlawful discrimination?',
       'Workplace. Topic:Does your workplace promote and practice the principles of employment equity?',
       'Workplace. Topic:Does your workplace value the diversity of its employees?',
       'Workplace. Topic:Would you recommend the Institute as an employer to others?'],
      dtype='object')
Out[8]:
Record ID Institute WorkArea CESSATION YEAR Reason for ceasing employment Contributing Factors. Career Move - Public Sector Contributing Factors. Career Move - Private Sector Contributing Factors. Career Move - Self-employment Contributing Factors. Ill Health Contributing Factors. Maternity/Family ... Contributing Factors. Study Contributing Factors. Travel Contributing Factors. Other Contributing Factors. NONE Gender. What is your Gender? CurrentAge. Current Age Employment Type. Employment Type Classification. Classification LengthofServiceOverall. Overall Length of Service at Institute (in years) LengthofServiceCurrent. Length of Service at current workplace (in years)
0 6.341330e+17 Southern Queensland Institute of TAFE Non-Delivery (corporate) 2010.0 Contract Expired NaN NaN NaN NaN NaN ... NaN NaN NaN NaN Female 26 30 Temporary Full-time Administration (AO) 1-2 1-2
1 6.341337e+17 Mount Isa Institute of TAFE Non-Delivery (corporate) 2010.0 Retirement - - - - - ... - Travel - - NaN NaN NaN NaN NaN NaN
2 6.341388e+17 Mount Isa Institute of TAFE Delivery (teaching) 2010.0 Retirement - - - - - ... - - - NONE NaN NaN NaN NaN NaN NaN
3 6.341399e+17 Mount Isa Institute of TAFE Non-Delivery (corporate) 2010.0 Resignation - - - - - ... - Travel - - NaN NaN NaN NaN NaN NaN
4 6.341466e+17 Southern Queensland Institute of TAFE Delivery (teaching) 2010.0 Resignation - Career Move - Private Sector - - - ... - - - - Male 41 45 Permanent Full-time Teacher (including LVT) 3-4 3-4

5 rows × 23 columns

Columns which will not be used in the analysis have been removed from each dataframe.

In [9]:
dete_survey_updated.columns = dete_survey_updated.columns.str.replace(" ", "_").str.lower()
In [10]:
# Update column names to match the names in dete_survey_updated
mapping = {'Record ID': 'id', 'CESSATION YEAR': 'cease_date', 'Reason for ceasing employment': 'separationtype', 'Gender. What is your Gender?': 'gender', 'CurrentAge. Current Age': 'age',
       'Employment Type. Employment Type': 'employment_status',
       'Classification. Classification': 'position',
       'LengthofServiceOverall. Overall Length of Service at Institute (in years)': 'institute_service',
       'LengthofServiceCurrent. Length of Service at current workplace (in years)': 'role_service'}

tafe_survey_updated = tafe_survey_updated.rename(mapping, axis = 1)

# Check that the specified column names were updated correctly
tafe_survey_updated.columns

tafe_survey_updated
Out[10]:
id Institute WorkArea cease_date separationtype Contributing Factors. Career Move - Public Sector Contributing Factors. Career Move - Private Sector Contributing Factors. Career Move - Self-employment Contributing Factors. Ill Health Contributing Factors. Maternity/Family ... Contributing Factors. Study Contributing Factors. Travel Contributing Factors. Other Contributing Factors. NONE gender age employment_status position institute_service role_service
0 6.341330e+17 Southern Queensland Institute of TAFE Non-Delivery (corporate) 2010.0 Contract Expired NaN NaN NaN NaN NaN ... NaN NaN NaN NaN Female 26 30 Temporary Full-time Administration (AO) 1-2 1-2
1 6.341337e+17 Mount Isa Institute of TAFE Non-Delivery (corporate) 2010.0 Retirement - - - - - ... - Travel - - NaN NaN NaN NaN NaN NaN
2 6.341388e+17 Mount Isa Institute of TAFE Delivery (teaching) 2010.0 Retirement - - - - - ... - - - NONE NaN NaN NaN NaN NaN NaN
3 6.341399e+17 Mount Isa Institute of TAFE Non-Delivery (corporate) 2010.0 Resignation - - - - - ... - Travel - - NaN NaN NaN NaN NaN NaN
4 6.341466e+17 Southern Queensland Institute of TAFE Delivery (teaching) 2010.0 Resignation - Career Move - Private Sector - - - ... - - - - Male 41 45 Permanent Full-time Teacher (including LVT) 3-4 3-4
5 6.341475e+17 Southern Queensland Institute of TAFE Delivery (teaching) 2010.0 Resignation - - - - - ... - - Other - Female 56 or older Contract/casual Teacher (including LVT) 7-10 7-10
6 6.341520e+17 Barrier Reef Institute of TAFE Non-Delivery (corporate) 2010.0 Resignation - Career Move - Private Sector - - Maternity/Family ... - - Other - Male 20 or younger Temporary Full-time Administration (AO) 3-4 3-4
7 6.341537e+17 Southern Queensland Institute of TAFE Delivery (teaching) 2010.0 Resignation - - - - - ... - - Other - Male 46 50 Permanent Full-time Teacher (including LVT) 3-4 3-4
8 6.341579e+17 Southern Queensland Institute of TAFE Delivery (teaching) 2009.0 Resignation - - - - - ... - - Other - Female 36 40 Temporary Full-time Tutor 3-4 3-4
9 6.341588e+17 Barrier Reef Institute of TAFE Non-Delivery (corporate) 2010.0 Resignation Career Move - Public Sector - - - - ... - - Other - Female 21 25 Permanent Full-time Administration (AO) 1-2 1-2
10 6.341588e+17 Barrier Reef Institute of TAFE Non-Delivery (corporate) 2010.0 Resignation Career Move - Public Sector - - - - ... - - - - Female 41 45 Temporary Part-time Administration (AO) Less than 1 year Less than 1 year
11 6.341719e+17 Barrier Reef Institute of TAFE Delivery (teaching) 2010.0 Retrenchment/ Redundancy NaN NaN NaN NaN NaN ... NaN NaN NaN NaN Male 56 or older Permanent Part-time Tutor 11-20 11-20
12 6.341719e+17 Barrier Reef Institute of TAFE Delivery (teaching) 2010.0 Retrenchment/ Redundancy NaN NaN NaN NaN NaN ... NaN NaN NaN NaN Male 51-55 Permanent Full-time Teacher (including LVT) 3-4 3-4
13 6.341725e+17 Barrier Reef Institute of TAFE Non-Delivery (corporate) 2010.0 Resignation - Career Move - Private Sector - - - ... - - - - Female 31 35 Temporary Full-time Administration (AO) 11-20 Less than 1 year
14 6.341726e+17 Central Queensland Institute of TAFE Delivery (teaching) 2010.0 Resignation - Career Move - Private Sector - - - ... - - - - Female 31 35 Permanent Part-time Teacher (including LVT) 7-10 7-10
15 6.341761e+17 Southern Queensland Institute of TAFE Non-Delivery (corporate) 2010.0 Resignation - - - - - ... - - Other - Female 46 50 Permanent Part-time Technical Officer (TO) 11-20 11-20
16 6.341770e+17 Brisbane North Institute of TAFE Non-Delivery (corporate) 2010.0 Resignation NaN NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
17 6.341771e+17 Brisbane North Institute of TAFE Non-Delivery (corporate) 2010.0 Resignation Career Move - Public Sector - - - - ... - - - - Female 31 35 Permanent Full-time Administration (AO) 7-10 1-2
18 6.341779e+17 Brisbane North Institute of TAFE Delivery (teaching) 2010.0 Resignation NaN NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
19 6.341820e+17 Southbank Institute of Technology Non-Delivery (corporate) 2010.0 Resignation - - - - - ... - - - NONE NaN NaN NaN NaN NaN NaN
20 6.341821e+17 Brisbane North Institute of TAFE Non-Delivery (corporate) 2010.0 Resignation - - - - - ... - - - - NaN NaN NaN NaN NaN NaN
21 6.341831e+17 Brisbane North Institute of TAFE Non-Delivery (corporate) 2010.0 Resignation - Career Move - Private Sector - - - ... - - - - NaN NaN NaN NaN NaN NaN
22 6.341847e+17 Southern Queensland Institute of TAFE Non-Delivery (corporate) 2010.0 Resignation Career Move - Public Sector - - - - ... - - Other - Male 26 30 Temporary Full-time Administration (AO) 5-6 5-6
23 6.341907e+17 Central Queensland Institute of TAFE Non-Delivery (corporate) 2010.0 Resignation - - - - - ... - - - - Female 26 30 Contract/casual Administration (AO) Less than 1 year Less than 1 year
24 6.341907e+17 Central Queensland Institute of TAFE Delivery (teaching) 2010.0 Resignation - Career Move - Private Sector - - - ... - - - - Female 31 35 Permanent Part-time Teacher (including LVT) 7-10 7-10
25 6.341909e+17 Brisbane North Institute of TAFE Non-Delivery (corporate) 2010.0 Contract Expired NaN NaN NaN NaN NaN ... NaN NaN NaN NaN Female 41 45 Temporary Full-time Administration (AO) Less than 1 year Less than 1 year
26 6.341934e+17 Mount Isa Institute of TAFE Non-Delivery (corporate) 2010.0 Resignation - Career Move - Private Sector - - - ... - - - - NaN NaN NaN NaN NaN NaN
27 6.341935e+17 Brisbane North Institute of TAFE Non-Delivery (corporate) 2010.0 Resignation Career Move - Public Sector - - - - ... - - - - Male 56 or older Temporary Full-time Administration (AO) More than 20 years 7-10
28 6.341951e+17 Central Queensland Institute of TAFE Delivery (teaching) 2010.0 Retirement - - - - - ... - - - - NaN NaN NaN NaN NaN NaN
29 6.341994e+17 Barrier Reef Institute of TAFE Non-Delivery (corporate) 2010.0 Resignation - Career Move - Private Sector - - - ... - - - - Female 31 35 Temporary Full-time Administration (AO) Less than 1 year Less than 1 year
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
672 6.350132e+17 Tropical North Institute of TAFE Non-Delivery (corporate) 2013.0 Retirement - - - Ill Health - ... - - - - NaN NaN NaN NaN NaN NaN
673 6.350166e+17 Southern Queensland Institute of TAFE Non-Delivery (corporate) 2013.0 Retrenchment/ Redundancy NaN NaN NaN NaN NaN ... NaN NaN NaN NaN Male 41 45 Temporary Full-time Administration (AO) 1-2 1-2
674 6.350170e+17 Sunshine Coast Institute of TAFE Non-Delivery (corporate) 2013.0 Retrenchment/ Redundancy NaN NaN NaN NaN NaN ... NaN NaN NaN NaN Female 51-55 Temporary Part-time Operational (OO) Less than 1 year Less than 1 year
675 6.350175e+17 Tropical North Institute of TAFE Delivery (teaching) 2013.0 Resignation - - - - - ... - - Other - Male 51-55 Temporary Full-time Teacher (including LVT) Less than 1 year Less than 1 year
676 6.350194e+17 Southern Queensland Institute of TAFE Non-Delivery (corporate) 2013.0 Resignation Career Move - Public Sector - - - - ... - - - - Female 41 45 Contract/casual Administration (AO) 1-2 Less than 1 year
677 6.350219e+17 Brisbane North Institute of TAFE Non-Delivery (corporate) 2013.0 Resignation - - - - - ... - - Other - Female 36 40 Temporary Full-time Administration (AO) Less than 1 year Less than 1 year
678 6.350253e+17 Brisbane North Institute of TAFE Non-Delivery (corporate) 2013.0 Resignation - Career Move - Private Sector - - - ... - - - - Male 51-55 Permanent Full-time Administration (AO) 3-4 3-4
679 6.350279e+17 Southern Queensland Institute of TAFE Non-Delivery (corporate) 2013.0 Resignation - - - Ill Health - ... - - - - Female 56 or older Temporary Part-time Operational (OO) 1-2 1-2
680 6.350306e+17 Brisbane North Institute of TAFE Delivery (teaching) 2013.0 Retrenchment/ Redundancy NaN NaN NaN NaN NaN ... NaN NaN NaN NaN Male 41 45 Permanent Full-time Administration (AO) 5-6 3-4
681 6.350314e+17 Central Queensland Institute of TAFE Non-Delivery (corporate) 2013.0 Resignation - - - - - ... - - Other - Female 26 30 Temporary Full-time Administration (AO) Less than 1 year Less than 1 year
682 6.350357e+17 Mount Isa Institute of TAFE Non-Delivery (corporate) 2013.0 Resignation Career Move - Public Sector - - - - ... - - - - Female 26 30 Permanent Part-time Administration (AO) Less than 1 year Less than 1 year
683 6.350374e+17 Central Queensland Institute of TAFE Non-Delivery (corporate) 2013.0 Resignation - - - - - ... - - Other - Female 41 45 Temporary Full-time Administration (AO) Less than 1 year Less than 1 year
684 6.350375e+17 Brisbane North Institute of TAFE Non-Delivery (corporate) 2013.0 Resignation Career Move - Public Sector - - - - ... - - - - Male 41 45 Contract/casual Administration (AO) 3-4 Less than 1 year
685 6.350402e+17 Brisbane North Institute of TAFE Non-Delivery (corporate) 2013.0 Resignation - - - - - ... - - - - Female 26 30 Temporary Full-time Technical Officer (TO) 1-2 1-2
686 6.350426e+17 Brisbane North Institute of TAFE Non-Delivery (corporate) 2013.0 Resignation Career Move - Public Sector - - - - ... - - - - Female 41 45 Temporary Full-time Administration (AO) 5-6 5-6
687 6.350428e+17 Southern Queensland Institute of TAFE Non-Delivery (corporate) 2013.0 Termination NaN NaN NaN NaN NaN ... NaN NaN NaN NaN Female 36 40 Temporary Full-time Administration (AO) 3-4 3-4
688 6.350479e+17 Tropical North Institute of TAFE Non-Delivery (corporate) 2013.0 Resignation - - - - - ... - - - NONE Female 46 50 Permanent Part-time Professional Officer (PO) 5-6 5-6
689 6.350480e+17 Central Queensland Institute of TAFE Delivery (teaching) 2013.0 Resignation - - - Ill Health - ... - - - - Male 41 45 Permanent Full-time Teacher (including LVT) Less than 1 year Less than 1 year
690 6.350496e+17 Southern Queensland Institute of TAFE Non-Delivery (corporate) 2013.0 Resignation - - - Ill Health - ... - - - - NaN NaN NaN NaN NaN NaN
691 6.350496e+17 Southern Queensland Institute of TAFE Non-Delivery (corporate) 2013.0 Resignation - - - Ill Health - ... - - - - Female 56 or older Permanent Part-time Operational (OO) 3-4 3-4
692 6.350592e+17 Mount Isa Institute of TAFE Delivery (teaching) 2013.0 Contract Expired NaN NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
693 6.350599e+17 Tropical North Institute of TAFE Non-Delivery (corporate) 2013.0 Resignation - Career Move - Private Sector - - - ... - - - - Female 26 30 Temporary Full-time Administration (AO) 1-2 1-2
694 6.350652e+17 Sunshine Coast Institute of TAFE Non-Delivery (corporate) 2013.0 Resignation - Career Move - Private Sector - - - ... - - - - NaN NaN NaN NaN NaN NaN
695 6.350653e+17 Southern Queensland Institute of TAFE Delivery (teaching) 2013.0 Contract Expired NaN NaN NaN NaN NaN ... NaN NaN NaN NaN Female 56 or older Temporary Part-time Teacher (including LVT) Less than 1 year Less than 1 year
696 6.350660e+17 Southern Queensland Institute of TAFE Non-Delivery (corporate) 2013.0 Resignation - Career Move - Private Sector - - - ... - - - - Male 21 25 Temporary Full-time Operational (OO) 5-6 5-6
697 6.350668e+17 Barrier Reef Institute of TAFE Delivery (teaching) 2013.0 Resignation Career Move - Public Sector - - - - ... - - - - Male 51-55 Temporary Full-time Teacher (including LVT) 1-2 1-2
698 6.350677e+17 Southern Queensland Institute of TAFE Non-Delivery (corporate) 2013.0 Resignation Career Move - Public Sector - - - - ... - - - - NaN NaN NaN NaN NaN NaN
699 6.350704e+17 Tropical North Institute of TAFE Delivery (teaching) 2013.0 Resignation - - - - - ... - - Other - Female 51-55 Permanent Full-time Teacher (including LVT) 5-6 1-2
700 6.350712e+17 Southbank Institute of Technology Non-Delivery (corporate) 2013.0 Contract Expired NaN NaN NaN NaN NaN ... NaN NaN NaN NaN Female 41 45 Temporary Full-time Professional Officer (PO) 1-2 1-2
701 6.350730e+17 Tropical North Institute of TAFE Non-Delivery (corporate) 2013.0 Resignation - - Career Move - Self-employment - - ... - Travel - - Female 26 30 Contract/casual Administration (AO) 3-4 1-2

702 rows × 23 columns

In [11]:
dete_survey_updated.head()
Out[11]:
id separationtype cease_date dete_start_date role_start_date position classification region business_unit employment_status ... work_life_balance workload none_of_the_above gender age aboriginal torres_strait south_sea disability nesb
0 1 Ill Health Retirement 08/2012 1984.0 2004.0 Public Servant A01-A04 Central Office Corporate Strategy and Peformance Permanent Full-time ... False False True Male 56-60 NaN NaN NaN NaN Yes
1 2 Voluntary Early Retirement (VER) 08/2012 NaN NaN Public Servant AO5-AO7 Central Office Corporate Strategy and Peformance Permanent Full-time ... False False False Male 56-60 NaN NaN NaN NaN NaN
2 3 Voluntary Early Retirement (VER) 05/2012 2011.0 2011.0 Schools Officer NaN Central Office Education Queensland Permanent Full-time ... False False True Male 61 or older NaN NaN NaN NaN NaN
3 4 Resignation-Other reasons 05/2012 2005.0 2006.0 Teacher Primary Central Queensland NaN Permanent Full-time ... False False False Female 36-40 NaN NaN NaN NaN NaN
4 5 Age Retirement 05/2012 1970.0 1989.0 Head of Curriculum/Head of Special Education NaN South East NaN Permanent Full-time ... True False False Female 61 or older NaN NaN NaN NaN NaN

5 rows × 35 columns

In [12]:
tafe_survey_updated.head()
Out[12]:
id Institute WorkArea cease_date separationtype Contributing Factors. Career Move - Public Sector Contributing Factors. Career Move - Private Sector Contributing Factors. Career Move - Self-employment Contributing Factors. Ill Health Contributing Factors. Maternity/Family ... Contributing Factors. Study Contributing Factors. Travel Contributing Factors. Other Contributing Factors. NONE gender age employment_status position institute_service role_service
0 6.341330e+17 Southern Queensland Institute of TAFE Non-Delivery (corporate) 2010.0 Contract Expired NaN NaN NaN NaN NaN ... NaN NaN NaN NaN Female 26 30 Temporary Full-time Administration (AO) 1-2 1-2
1 6.341337e+17 Mount Isa Institute of TAFE Non-Delivery (corporate) 2010.0 Retirement - - - - - ... - Travel - - NaN NaN NaN NaN NaN NaN
2 6.341388e+17 Mount Isa Institute of TAFE Delivery (teaching) 2010.0 Retirement - - - - - ... - - - NONE NaN NaN NaN NaN NaN NaN
3 6.341399e+17 Mount Isa Institute of TAFE Non-Delivery (corporate) 2010.0 Resignation - - - - - ... - Travel - - NaN NaN NaN NaN NaN NaN
4 6.341466e+17 Southern Queensland Institute of TAFE Delivery (teaching) 2010.0 Resignation - Career Move - Private Sector - - - ... - - - - Male 41 45 Permanent Full-time Teacher (including LVT) 3-4 3-4

5 rows × 23 columns

Columns have been renamed to make analysis easier and to reduce the liklihood of mistakes. Also, columns which are not relevent to the analysis have been removed.

Filter the datasets for entries for Resignation

The stakeholders are interested in why employees resign, therefore only keep entries relating to resignations.

In [13]:
dete_survey_updated['separationtype'].value_counts()
Out[13]:
Age Retirement                          285
Resignation-Other reasons               150
Resignation-Other employer               91
Resignation-Move overseas/interstate     70
Voluntary Early Retirement (VER)         67
Ill Health Retirement                    61
Other                                    49
Contract Expired                         34
Termination                              15
Name: separationtype, dtype: int64
In [14]:
tafe_survey_updated['separationtype'].value_counts()
Out[14]:
Resignation                 340
Contract Expired            127
Retrenchment/ Redundancy    104
Retirement                   82
Transfer                     25
Termination                  23
Name: separationtype, dtype: int64
In [15]:
dete_resig_logic = dete_survey_updated['separationtype'].str.contains("Resignation")
#print(dete_resig_logic)
dete_resignations = dete_survey_updated[dete_resig_logic].copy()
dete_resignations.head()
Out[15]:
id separationtype cease_date dete_start_date role_start_date position classification region business_unit employment_status ... work_life_balance workload none_of_the_above gender age aboriginal torres_strait south_sea disability nesb
3 4 Resignation-Other reasons 05/2012 2005.0 2006.0 Teacher Primary Central Queensland NaN Permanent Full-time ... False False False Female 36-40 NaN NaN NaN NaN NaN
5 6 Resignation-Other reasons 05/2012 1994.0 1997.0 Guidance Officer NaN Central Office Education Queensland Permanent Full-time ... False False False Female 41-45 NaN NaN NaN NaN NaN
8 9 Resignation-Other reasons 07/2012 2009.0 2009.0 Teacher Secondary North Queensland NaN Permanent Full-time ... False False False Female 31-35 NaN NaN NaN NaN NaN
9 10 Resignation-Other employer 2012 1997.0 2008.0 Teacher Aide NaN NaN NaN Permanent Part-time ... False False False Female 46-50 NaN NaN NaN NaN NaN
11 12 Resignation-Move overseas/interstate 2012 2009.0 2009.0 Teacher Secondary Far North Queensland NaN Permanent Full-time ... False False False Male 31-35 NaN NaN NaN NaN NaN

5 rows × 35 columns

In [16]:
tafe_resig_logic = tafe_survey_updated['separationtype'].str.contains("Resignation", na=False)
#print(tafe_resig_logic)
tafe_resignations = tafe_survey_updated[tafe_resig_logic].copy()
tafe_resignations.head()
Out[16]:
id Institute WorkArea cease_date separationtype Contributing Factors. Career Move - Public Sector Contributing Factors. Career Move - Private Sector Contributing Factors. Career Move - Self-employment Contributing Factors. Ill Health Contributing Factors. Maternity/Family ... Contributing Factors. Study Contributing Factors. Travel Contributing Factors. Other Contributing Factors. NONE gender age employment_status position institute_service role_service
3 6.341399e+17 Mount Isa Institute of TAFE Non-Delivery (corporate) 2010.0 Resignation - - - - - ... - Travel - - NaN NaN NaN NaN NaN NaN
4 6.341466e+17 Southern Queensland Institute of TAFE Delivery (teaching) 2010.0 Resignation - Career Move - Private Sector - - - ... - - - - Male 41 45 Permanent Full-time Teacher (including LVT) 3-4 3-4
5 6.341475e+17 Southern Queensland Institute of TAFE Delivery (teaching) 2010.0 Resignation - - - - - ... - - Other - Female 56 or older Contract/casual Teacher (including LVT) 7-10 7-10
6 6.341520e+17 Barrier Reef Institute of TAFE Non-Delivery (corporate) 2010.0 Resignation - Career Move - Private Sector - - Maternity/Family ... - - Other - Male 20 or younger Temporary Full-time Administration (AO) 3-4 3-4
7 6.341537e+17 Southern Queensland Institute of TAFE Delivery (teaching) 2010.0 Resignation - - - - - ... - - Other - Male 46 50 Permanent Full-time Teacher (including LVT) 3-4 3-4

5 rows × 23 columns

Cleaning columns containing start and end dates

The dete_resignations dataset contains information on cease_date but the format is not consistent. Explore the date data in both datasets and ensure it is consistent.

In [17]:
print(dete_resignations["cease_date"].value_counts(dropna = False))
#print(dete_resignations[[770])
#pattern = r"(20[0-1][0-9])"
pattern = r"(?P<Year>20[0-9]{2})"
#r"([1-2][0-9]{3})"
dete_resignations['cease_date'] = dete_resignations['cease_date'].str.extract(pattern, expand = True).astype(float)
#print(resig_years)

print(dete_resignations.head())
2012       126
2013        74
01/2014     22
12/2013     17
06/2013     14
09/2013     11
NaN         11
11/2013      9
07/2013      9
10/2013      6
08/2013      4
05/2012      2
05/2013      2
09/2010      1
2010         1
07/2006      1
07/2012      1
Name: cease_date, dtype: int64
    id                        separationtype  cease_date  dete_start_date  \
3    4             Resignation-Other reasons      2012.0           2005.0   
5    6             Resignation-Other reasons      2012.0           1994.0   
8    9             Resignation-Other reasons      2012.0           2009.0   
9   10            Resignation-Other employer      2012.0           1997.0   
11  12  Resignation-Move overseas/interstate      2012.0           2009.0   

    role_start_date          position classification                region  \
3            2006.0           Teacher        Primary    Central Queensland   
5            1997.0  Guidance Officer            NaN        Central Office   
8            2009.0           Teacher      Secondary      North Queensland   
9            2008.0      Teacher Aide            NaN                   NaN   
11           2009.0           Teacher      Secondary  Far North Queensland   

           business_unit    employment_status  ...   work_life_balance  \
3                    NaN  Permanent Full-time  ...               False   
5   Education Queensland  Permanent Full-time  ...               False   
8                    NaN  Permanent Full-time  ...               False   
9                    NaN  Permanent Part-time  ...               False   
11                   NaN  Permanent Full-time  ...               False   

    workload  none_of_the_above  gender    age  aboriginal  torres_strait  \
3      False              False  Female  36-40         NaN            NaN   
5      False              False  Female  41-45         NaN            NaN   
8      False              False  Female  31-35         NaN            NaN   
9      False              False  Female  46-50         NaN            NaN   
11     False              False    Male  31-35         NaN            NaN   

    south_sea  disability  nesb  
3         NaN         NaN   NaN  
5         NaN         NaN   NaN  
8         NaN         NaN   NaN  
9         NaN         NaN   NaN  
11        NaN         NaN   NaN  

[5 rows x 35 columns]

The cease_date column in the dete_resignations data frame has been extracted and cleaned to only include the year. Resignations occur between the years 2006 and 2013 with the majority occurring in 2012 and 2013. There are 11 instances where the cease_date was not recorded.

In [18]:
dete_resignations['dete_start_date'].value_counts(dropna=False).sort_index(ascending=True)
Out[18]:
 1963.0     1
 1971.0     1
 1972.0     1
 1973.0     1
 1974.0     2
 1975.0     1
 1976.0     2
 1977.0     1
 1980.0     5
 1982.0     1
 1983.0     2
 1984.0     1
 1985.0     3
 1986.0     3
 1987.0     1
 1988.0     4
 1989.0     4
 1990.0     5
 1991.0     4
 1992.0     6
 1993.0     5
 1994.0     6
 1995.0     4
 1996.0     6
 1997.0     5
 1998.0     6
 1999.0     8
 2000.0     9
 2001.0     3
 2002.0     6
 2003.0     6
 2004.0    14
 2005.0    15
 2006.0    13
 2007.0    21
 2008.0    22
 2009.0    13
 2010.0    17
 2011.0    24
 2012.0    21
 2013.0    10
NaN        28
Name: dete_start_date, dtype: int64

The dete_start_date varies from 1963 to 2013. Although 1963 is quite early, there is nothing to suggest that it is an error.

In [19]:
tafe_resignations['cease_date'].value_counts(dropna = False).sort_index(ascending = True)
Out[19]:
 2009.0      2
 2010.0     68
 2011.0    116
 2012.0     94
 2013.0     55
NaN          5
Name: cease_date, dtype: int64

In the tafe_resignations dataset the cease_date column contains the years 2009-2013 and also 5 NaN entries.

Compared with the dete_resignations dataset, the cease_date column is quite similar except that it contains an entry for 2006 so it may be worth dropping this entry.

Create Length of Service Column

The tafe_resignations dataset contains a length of service column called 'institute service'. Create the equivalent column in the dete_resignations dataset to enable comparisons to be made.

In [20]:
dete_resignations['institute_service'] = dete_resignations['cease_date'] - dete_resignations['dete_start_date']

print(dete_resignations.head())
    id                        separationtype  cease_date  dete_start_date  \
3    4             Resignation-Other reasons      2012.0           2005.0   
5    6             Resignation-Other reasons      2012.0           1994.0   
8    9             Resignation-Other reasons      2012.0           2009.0   
9   10            Resignation-Other employer      2012.0           1997.0   
11  12  Resignation-Move overseas/interstate      2012.0           2009.0   

    role_start_date          position classification                region  \
3            2006.0           Teacher        Primary    Central Queensland   
5            1997.0  Guidance Officer            NaN        Central Office   
8            2009.0           Teacher      Secondary      North Queensland   
9            2008.0      Teacher Aide            NaN                   NaN   
11           2009.0           Teacher      Secondary  Far North Queensland   

           business_unit    employment_status        ...          workload  \
3                    NaN  Permanent Full-time        ...             False   
5   Education Queensland  Permanent Full-time        ...             False   
8                    NaN  Permanent Full-time        ...             False   
9                    NaN  Permanent Part-time        ...             False   
11                   NaN  Permanent Full-time        ...             False   

    none_of_the_above  gender    age  aboriginal  torres_strait  south_sea  \
3               False  Female  36-40         NaN            NaN        NaN   
5               False  Female  41-45         NaN            NaN        NaN   
8               False  Female  31-35         NaN            NaN        NaN   
9               False  Female  46-50         NaN            NaN        NaN   
11              False    Male  31-35         NaN            NaN        NaN   

    disability  nesb  institute_service  
3          NaN   NaN                7.0  
5          NaN   NaN               18.0  
8          NaN   NaN                3.0  
9          NaN   NaN               15.0  
11         NaN   NaN                3.0  

[5 rows x 36 columns]

A new column institute_service has been created in the dete_resignations dataset to enable direct comparison with a column in the tafe_resignations dataset.

Explore and clean 'dissatisfaction' columns

In order to be able to compare employees 'dissatisfaction' across the two datasets, the columns relating to dissatisfaction need to be simplified. To do this, if an entry in either dataset contains values which indicate dissatisfaction, then a new column will be marked 'True' for dissatifcation. This way, both datasets will end up with a column containing binary/logic data i.e. True/False and thus can be compared.

In [21]:
print(tafe_resignations['Contributing Factors. Dissatisfaction'].value_counts(dropna = False))
print(tafe_resignations['Contributing Factors. Job Dissatisfaction'].value_counts(dropna = False))
print(tafe_resignations['Contributing Factors. Dissatisfaction'].head(50))
-                                         277
Contributing Factors. Dissatisfaction      55
NaN                                         8
Name: Contributing Factors. Dissatisfaction, dtype: int64
-                      270
Job Dissatisfaction     62
NaN                      8
Name: Contributing Factors. Job Dissatisfaction, dtype: int64
3                                          -
4                                          -
5                                          -
6                                          -
7                                          -
8                                          -
9                                          -
10                                         -
13                                         -
14    Contributing Factors. Dissatisfaction 
15                                         -
16                                       NaN
17                                         -
18                                       NaN
19                                         -
20    Contributing Factors. Dissatisfaction 
21                                         -
22                                         -
23                                         -
24                                         -
26    Contributing Factors. Dissatisfaction 
27                                         -
29                                         -
32                                         -
36                                         -
37                                         -
39                                         -
40    Contributing Factors. Dissatisfaction 
41                                         -
42                                         -
45                                         -
46                                         -
47    Contributing Factors. Dissatisfaction 
49                                         -
51                                       NaN
52                                         -
53                                         -
54    Contributing Factors. Dissatisfaction 
55                                         -
56                                         -
58                                         -
63                                         -
64    Contributing Factors. Dissatisfaction 
65    Contributing Factors. Dissatisfaction 
66                                         -
69                                         -
71    Contributing Factors. Dissatisfaction 
72                                         -
74                                         -
79                                         -
Name: Contributing Factors. Dissatisfaction, dtype: object
In [36]:
def update_vals(value):
        if pd.isnull(value) == True:
            return(np.nan)
        elif value == '-':
            return(False)
        else:
            return(True)
        
tester = ['NaN', 'diss', '-', '-', 'diss', 'NaN']

cols_to_update = ['Contributing Factors. Dissatisfaction', 'Contributing Factors. Job Dissatisfaction']

#print(tafe_resignations[cols_to_update].head())

#tafe_resignations[cols_to_update] = 
tafe_factors = tafe_resignations[cols_to_update].applymap(update_vals)

tafe_resignations['dissatisfied'] = tafe_factors.any(axis=1, skipna = False)

print(tafe_resignations.head())

print(tafe_resignations['dissatisfied'].value_counts(dropna = False))

tafe_resignations_up = tafe_resignations.copy()
             id                              Institute  \
3  6.341399e+17            Mount Isa Institute of TAFE   
4  6.341466e+17  Southern Queensland Institute of TAFE   
5  6.341475e+17  Southern Queensland Institute of TAFE   
6  6.341520e+17         Barrier Reef Institute of TAFE   
7  6.341537e+17  Southern Queensland Institute of TAFE   

                   WorkArea  cease_date separationtype  \
3  Non-Delivery (corporate)      2010.0    Resignation   
4       Delivery (teaching)      2010.0    Resignation   
5       Delivery (teaching)      2010.0    Resignation   
6  Non-Delivery (corporate)      2010.0    Resignation   
7       Delivery (teaching)      2010.0    Resignation   

  Contributing Factors. Career Move - Public Sector   \
3                                                  -   
4                                                  -   
5                                                  -   
6                                                  -   
7                                                  -   

  Contributing Factors. Career Move - Private Sector   \
3                                                  -    
4                       Career Move - Private Sector    
5                                                  -    
6                       Career Move - Private Sector    
7                                                  -    

  Contributing Factors. Career Move - Self-employment  \
3                                                  -    
4                                                  -    
5                                                  -    
6                                                  -    
7                                                  -    

  Contributing Factors. Ill Health Contributing Factors. Maternity/Family  \
3                                -                                      -   
4                                -                                      -   
5                                -                                      -   
6                                -                       Maternity/Family   
7                                -                                      -   

      ...      Contributing Factors. Travel Contributing Factors. Other  \
3     ...                            Travel                           -   
4     ...                                 -                           -   
5     ...                                 -                       Other   
6     ...                                 -                       Other   
7     ...                                 -                       Other   

  Contributing Factors. NONE  gender            age    employment_status  \
3                          -     NaN            NaN                  NaN   
4                          -    Male         41  45  Permanent Full-time   
5                          -  Female    56 or older      Contract/casual   
6                          -    Male  20 or younger  Temporary Full-time   
7                          -    Male         46  50  Permanent Full-time   

                  position institute_service role_service dissatisfied  
3                      NaN               NaN          NaN        False  
4  Teacher (including LVT)               3-4          3-4        False  
5  Teacher (including LVT)              7-10         7-10        False  
6      Administration (AO)               3-4          3-4        False  
7  Teacher (including LVT)               3-4          3-4        False  

[5 rows x 24 columns]
False    241
True      91
NaN        8
Name: dissatisfied, dtype: int64
In [73]:
dete_cols = ['job_dissatisfaction', 'dissatisfaction_with_the_department', 'physical_work_environment',
             'lack_of_recognition', 'lack_of_job_security', 'work_location', 
             'employment_conditions', 'work_life_balance', 'workload']

diss_cols = dete_resignations[dete_cols]

print(diss_cols.head(10))

dete_resignations['dissatisfied'] = diss_cols.any(axis=1, skipna = False)


print(dete_resignations['dissatisfied'].value_counts(dropna = False))
print(dete_resignations.head(10))

dete_resignations_up = dete_resignations.copy()
    job_dissatisfaction  dissatisfaction_with_the_department  \
3                 False                                False   
5                 False                                False   
8                 False                                False   
9                  True                                 True   
11                False                                False   
12                False                                False   
14                 True                                 True   
16                False                                False   
20                False                                False   
21                False                                False   

    physical_work_environment  lack_of_recognition  lack_of_job_security  \
3                       False                False                 False   
5                       False                False                 False   
8                       False                False                 False   
9                       False                False                 False   
11                      False                False                 False   
12                      False                False                 False   
14                      False                False                 False   
16                      False                 True                 False   
20                      False                False                 False   
21                      False                False                 False   

    work_location  employment_conditions  work_life_balance  workload  
3           False                  False              False     False  
5           False                   True              False     False  
8           False                  False              False     False  
9           False                  False              False     False  
11          False                  False              False     False  
12          False                  False              False     False  
14          False                  False              False     False  
16          False                  False              False     False  
20          False                  False              False     False  
21          False                  False              False     False  
False    162
True     149
Name: dissatisfied, dtype: int64
    id                        separationtype  cease_date  dete_start_date  \
3    4             Resignation-Other reasons      2012.0           2005.0   
5    6             Resignation-Other reasons      2012.0           1994.0   
8    9             Resignation-Other reasons      2012.0           2009.0   
9   10            Resignation-Other employer      2012.0           1997.0   
11  12  Resignation-Move overseas/interstate      2012.0           2009.0   
12  13             Resignation-Other reasons      2012.0           1998.0   
14  15            Resignation-Other employer      2012.0           2007.0   
16  17             Resignation-Other reasons      2012.0              NaN   
20  21            Resignation-Other employer      2012.0           1982.0   
21  22             Resignation-Other reasons      2012.0           1980.0   

    role_start_date          position classification  \
3            2006.0           Teacher        Primary   
5            1997.0  Guidance Officer            NaN   
8            2009.0           Teacher      Secondary   
9            2008.0      Teacher Aide            NaN   
11           2009.0           Teacher      Secondary   
12           1998.0           Teacher        Primary   
14           2010.0           Teacher      Secondary   
16              NaN      Teacher Aide            NaN   
20           1982.0           Teacher      Secondary   
21           2009.0           Cleaner            NaN   

                      region         business_unit    employment_status  \
3         Central Queensland                   NaN  Permanent Full-time   
5             Central Office  Education Queensland  Permanent Full-time   
8           North Queensland                   NaN  Permanent Full-time   
9                        NaN                   NaN  Permanent Part-time   
11      Far North Queensland                   NaN  Permanent Full-time   
12      Far North Queensland                   NaN  Permanent Full-time   
14        Central Queensland                   NaN  Permanent Full-time   
16                South East                   NaN  Permanent Part-time   
20        Central Queensland                   NaN  Permanent Full-time   
21  Darling Downs South West                   NaN  Permanent Part-time   

        ...       none_of_the_above  gender          age  aboriginal  \
3       ...                   False  Female        36-40         NaN   
5       ...                   False  Female        41-45         NaN   
8       ...                   False  Female        31-35         NaN   
9       ...                   False  Female        46-50         NaN   
11      ...                   False    Male        31-35         NaN   
12      ...                   False  Female        36-40         NaN   
14      ...                   False    Male        31-35         NaN   
16      ...                   False    Male  61 or older         NaN   
20      ...                    True    Male        56-60         NaN   
21      ...                   False  Female        51-55         NaN   

    torres_strait  south_sea  disability  nesb  institute_service  \
3             NaN        NaN         NaN   NaN                7.0   
5             NaN        NaN         NaN   NaN               18.0   
8             NaN        NaN         NaN   NaN                3.0   
9             NaN        NaN         NaN   NaN               15.0   
11            NaN        NaN         NaN   NaN                3.0   
12            NaN        NaN         NaN   NaN               14.0   
14            NaN        NaN         NaN   NaN                5.0   
16            NaN        NaN         NaN   NaN                NaN   
20            NaN        NaN         NaN   NaN               30.0   
21            NaN        NaN         NaN   NaN               32.0   

    dissatisfied  
3          False  
5           True  
8          False  
9           True  
11         False  
12         False  
14          True  
16          True  
20         False  
21         False  

[10 rows x 37 columns]

Combine the datasets

In [144]:
#create a column containing the institute to enable to results to be compared by institute later on

dete_resignations_up['institute'] = 'DETE'

tafe_resignations_up['institute'] = 'TAFE'

combined = pd.concat([dete_resignations_up, tafe_resignations_up], ignore_index=True)
#print(combined.head())

combined_updated = combined.dropna(thresh = 500, axis =1).copy()
print(combined_updated.head())
print(combined_updated.shape)
     age  cease_date dissatisfied    employment_status  gender    id  \
0  36-40      2012.0        False  Permanent Full-time  Female   4.0   
1  41-45      2012.0         True  Permanent Full-time  Female   6.0   
2  31-35      2012.0        False  Permanent Full-time  Female   9.0   
3  46-50      2012.0         True  Permanent Part-time  Female  10.0   
4  31-35      2012.0        False  Permanent Full-time    Male  12.0   

  institute institute_service          position  \
0      DETE                 7           Teacher   
1      DETE                18  Guidance Officer   
2      DETE                 3           Teacher   
3      DETE                15      Teacher Aide   
4      DETE                 3           Teacher   

                         separationtype  
0             Resignation-Other reasons  
1             Resignation-Other reasons  
2             Resignation-Other reasons  
3            Resignation-Other employer  
4  Resignation-Move overseas/interstate  
(651, 10)

Clean institute service column

In [75]:
#print(combined_updated['institute_service'].value_counts(dropna=False))

def update_service(x):
    if x in ['Less than 1 year', '1-2', 0, 1, 2]:
        return("< 3 yrs")
    elif x in ['3-4', '5-6', 3, 4, 5, 6]:
        return("3-6 yrs")
    elif x in ['7-10', 7, 8, 9, 10]:
        return("7-10 yrs")
    elif x in ['11-20', 'More than 20 years']:
        return("11+ yrs")
    elif x > 10:
        return("11+ yrs")
    else:
        return(x)
        
combined_updated['institute_service'].head()
        
combined_updated['service_cat'] = combined_updated['institute_service'].apply(update_service)

print(combined_updated.head())

print(combined_updated['service_cat'].value_counts(dropna = False))
     age  cease_date dissatisfied    employment_status  gender    id  \
0  36-40      2012.0        False  Permanent Full-time  Female   4.0   
1  41-45      2012.0         True  Permanent Full-time  Female   6.0   
2  31-35      2012.0        False  Permanent Full-time  Female   9.0   
3  46-50      2012.0         True  Permanent Part-time  Female  10.0   
4  31-35      2012.0        False  Permanent Full-time    Male  12.0   

  institute institute_service          position  \
0      DETE                 7           Teacher   
1      DETE                18  Guidance Officer   
2      DETE                 3           Teacher   
3      DETE                15      Teacher Aide   
4      DETE                 3           Teacher   

                         separationtype service_cat  
0             Resignation-Other reasons    7-10 yrs  
1             Resignation-Other reasons     11+ yrs  
2             Resignation-Other reasons     3-6 yrs  
3            Resignation-Other employer     11+ yrs  
4  Resignation-Move overseas/interstate     3-6 yrs  
< 3 yrs     193
3-6 yrs     172
11+ yrs     136
NaN          88
7-10 yrs     62
Name: service_cat, dtype: int64
In [77]:
combined_updated['dissatisfied'].value_counts(dropna= False)
Out[77]:
False    403
True     240
NaN        8
Name: dissatisfied, dtype: int64
In [81]:
combined_updated['dissatisfied'] = combined_updated['dissatisfied'].fillna(False)
In [83]:
combined_updated['dissatisfied'].value_counts(dropna= False)
Out[83]:
False    411
True     240
Name: dissatisfied, dtype: int64

Comparing dissatisfaction by no. of years service

In [88]:
combined_pivot = combined_updated.pivot_table(index = 'service_cat', values = 'dissatisfied')
In [92]:
%matplotlib inline
combined_pivot.plot(kind = "bar", title = "Dissatisfaction by No. Years Service", xlim = (0,1))
Out[92]:
<matplotlib.axes._subplots.AxesSubplot at 0x7fe84efc7828>

The plot shows that the group most likely to be dissatisfied is those working in the organisation between 7 and 10 years. Those working less than three years are least likely to have left due to dissatisfaction.

Comparing dissatisfaction from DETE and TAFE

Although the first part of this project looked at both institutes combined together, it would be interesting to know which institute has more employees leaving due to dissatisfaction.

In [98]:
#print(combined_updated.head())

pivot_by_institute = combined_updated.pivot_table(index = 'institute', values = 'dissatisfied')
#print(pivot_by_institute)
pivot_by_institute.plot(kind = "bar", title = "Dissatisfaction by Institute", xlim = (0,1))
           dissatisfied
institute              
DETE           0.479100
TAFE           0.267647
Out[98]:
<matplotlib.axes._subplots.AxesSubplot at 0x7fe84ce0e780>

This is interesting - out of all the employees that leave DETE, nearly 50% leave because they are dissatisfied. Out of the employees that leave TAFE, 27% leave because they are dissatisfied.

Dissatisfaction by Gender

In [102]:
#print(combined_updated.head())

pivot_by_gender = combined_updated.pivot_table(index = 'gender', values = 'dissatisfied')
print(pivot_by_institute)
pivot_by_gender.plot(kind = "bar", title = "Dissatisfaction by Gender", xlim = (0,1))
           dissatisfied
institute              
DETE           0.479100
TAFE           0.267647
Out[102]:
<matplotlib.axes._subplots.AxesSubplot at 0x7fe84cc17550>

There isn't a great deal of difference here. Slightly more men cite dissatisfaction as a reason for leaving than women.

Dissatisfaction by 'cease date'

In [110]:
#print(combined_updated.head())

print(combined_updated['cease_date'].value_counts(dropna=False))

pivot_by_ceasedate = combined_updated[combined_updated['cease_date'] > 2009].pivot_table(index = 'cease_date', values = 'dissatisfied')
pivot_by_ceasedate.plot(kind = "barh", title = "Dissatisfaction by cease date", xlim = (0,0.5))
 2012.0    223
 2013.0    201
 2011.0    116
 2010.0     70
 2014.0     22
NaN         16
 2009.0      2
 2006.0      1
Name: cease_date, dtype: int64
Out[110]:
<matplotlib.axes._subplots.AxesSubplot at 0x7fe84c968128>

It appears that of those that left in 2010 and 2011, a smaller proportion expressed disatisfaction compared to those who left in 2012-2014. It would be interesting to know if anything caused this change i.e. a change of management, recession, etc.

There is only a couple of data entries for the years 2006 and 2009 so these entries have been removed.

Dissatisfaction by Age Group

It would be good to identify whether certain ages of people leaving were more likely to express dissatisfaction. However, first the age column requires cleaning.

In [140]:
print(combined_updated['age'].value_counts(dropna=False))
updated_age = combined_updated['age'].str.replace(" or older", "").str.replace("  ", "-").str.replace(" ", "-")

def change_age(x):
    if x in ['56', '56-60', '61']:
        return "56+"
    if x == "20-or-younger":
        return "under 21"
    else:
        return x
    
updated_age_2 = updated_age.apply(change_age)

print(updated_age_2.value_counts(dropna=False))

combined_updated['age_updated'] = updated_age_2

print(combined_updated.head(20))
51-55            71
NaN              55
41-45            48
41  45           45
46-50            42
36-40            41
46  50           39
26-30            35
21  25           33
26  30           32
31  35           32
36  40           32
56 or older      29
21-25            29
31-35            29
56-60            26
61 or older      23
20 or younger    10
Name: age, dtype: int64
41-45       93
46-50       81
56+         78
36-40       73
51-55       71
26-30       67
21-25       62
31-35       61
NaN         55
under 21    10
Name: age, dtype: int64
            age  cease_date  dissatisfied    employment_status  gender    id  \
0         36-40      2012.0         False  Permanent Full-time  Female   4.0   
1         41-45      2012.0          True  Permanent Full-time  Female   6.0   
2         31-35      2012.0         False  Permanent Full-time  Female   9.0   
3         46-50      2012.0          True  Permanent Part-time  Female  10.0   
4         31-35      2012.0         False  Permanent Full-time    Male  12.0   
5         36-40      2012.0         False  Permanent Full-time  Female  13.0   
6         31-35      2012.0          True  Permanent Full-time    Male  15.0   
7   61 or older      2012.0          True  Permanent Part-time    Male  17.0   
8         56-60      2012.0         False  Permanent Full-time    Male  21.0   
9         51-55      2012.0         False  Permanent Part-time  Female  22.0   
10        46-50      2012.0          True  Permanent Part-time  Female  23.0   
11  61 or older      2012.0          True  Permanent Full-time  Female  24.0   
12        41-45      2012.0          True  Permanent Part-time  Female  26.0   
13        21-25      2012.0         False  Permanent Full-time  Female  28.0   
14        36-40      2012.0          True  Permanent Full-time    Male  34.0   
15  61 or older      2012.0          True  Permanent Part-time    Male  35.0   
16        21-25      2012.0         False  Temporary Part-time  Female  38.0   
17        21-25      2012.0          True  Permanent Full-time  Female  40.0   
18        56-60      2012.0         False  Permanent Full-time    Male  41.0   
19        51-55      2012.0          True  Permanent Full-time  Female  42.0   

   institute institute_service                                      position  \
0       DETE                 7                                       Teacher   
1       DETE                18                              Guidance Officer   
2       DETE                 3                                       Teacher   
3       DETE                15                                  Teacher Aide   
4       DETE                 3                                       Teacher   
5       DETE                14                                       Teacher   
6       DETE                 5                                       Teacher   
7       DETE               NaN                                  Teacher Aide   
8       DETE                30                                       Teacher   
9       DETE                32                                       Cleaner   
10      DETE                15                   School Administrative Staff   
11      DETE                39                                       Teacher   
12      DETE                17                                       Teacher   
13      DETE                 7                                Public Servant   
14      DETE                 9                                       Teacher   
15      DETE                 6                                       Cleaner   
16      DETE                 1                                  Teacher Aide   
17      DETE               NaN                                       Teacher   
18      DETE                35                                       Teacher   
19      DETE                38  Head of Curriculum/Head of Special Education   

                          separationtype service_cat age_updated  
0              Resignation-Other reasons    7-10 yrs       36-40  
1              Resignation-Other reasons     11+ yrs       41-45  
2              Resignation-Other reasons     3-6 yrs       31-35  
3             Resignation-Other employer     11+ yrs       46-50  
4   Resignation-Move overseas/interstate     3-6 yrs       31-35  
5              Resignation-Other reasons     11+ yrs       36-40  
6             Resignation-Other employer     3-6 yrs       31-35  
7              Resignation-Other reasons         NaN         56+  
8             Resignation-Other employer     11+ yrs         56+  
9              Resignation-Other reasons     11+ yrs       51-55  
10             Resignation-Other reasons     11+ yrs       46-50  
11             Resignation-Other reasons     11+ yrs         56+  
12             Resignation-Other reasons     11+ yrs       41-45  
13            Resignation-Other employer    7-10 yrs       21-25  
14             Resignation-Other reasons    7-10 yrs       36-40  
15             Resignation-Other reasons     3-6 yrs         56+  
16             Resignation-Other reasons     < 3 yrs       21-25  
17  Resignation-Move overseas/interstate         NaN       21-25  
18            Resignation-Other employer     11+ yrs         56+  
19             Resignation-Other reasons     11+ yrs       51-55  
In [142]:
print(combined_updated['age_updated'].value_counts(dropna=False))

pivot_by_age = combined_updated.pivot_table(index = 'age_updated', values = 'dissatisfied')
pivot_by_age.plot(kind = "barh", title = "Dissatisfaction by Age", xlim = (0,0.5))
41-45       93
46-50       81
56+         78
36-40       73
51-55       71
26-30       67
21-25       62
31-35       61
NaN         55
under 21    10
Name: age_updated, dtype: int64
Out[142]:
<matplotlib.axes._subplots.AxesSubplot at 0x7fe84c6c12e8>

The age groups that express the most dissatisfaction are the 26-30 age group and the 51-55 and 56+ age groups. The under 21 age group expresses the least dissatisfaction when leaving a job.

In [ ]: