According to this article, data cleaning and organizing constitutes 57% of the total weight when it comes to the part of the data science.
Importing the required libraries.
Getting the data-set from a different source (Kaggle) and displaying the dataset.
Removing the unused or irrelevant columns.
Renaming the column names as per our convenience.
Replacing the value of the rows and make it more meaningful.
Even though this tutorial is small, but it’s a good way to start on small things and get our hands dirty later on. I will make sure that everyone with no prior experience in python programming or don’t know what is data science or data cleaning can easily understand this tutorial. I'm not very good at python in the first place, so even for me, this was a good place to start. One thing with python is that the code is self-explanatory, your focus should not be what the code does, because the code pretty much says what it does, rather you should tell why did you choose to do this, the “why” factor is important than the “what” factor.
# Importing the necessary libraries.
import pandas as pd
import numpy as np
This step involves getting the data-set from a different source, and the link for the data-set is provided below.
# Reading a CSV file
df = pd.read_csv("heart.csv")
df.head(5)
age | sex | cp | trestbps | chol | fbs | restecg | thalach | exang | oldpeak | slope | ca | thal | target | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 63 | 1 | 3 | 145 | 233 | 1 | 0 | 150 | 0 | 2.3 | 0 | 0 | 1 | 1 |
1 | 37 | 1 | 2 | 130 | 250 | 0 | 1 | 187 | 0 | 3.5 | 0 | 0 | 2 | 1 |
2 | 41 | 0 | 1 | 130 | 204 | 0 | 0 | 172 | 0 | 1.4 | 2 | 0 | 2 | 1 |
3 | 56 | 1 | 1 | 120 | 236 | 0 | 1 | 178 | 0 | 0.8 | 2 | 0 | 2 | 1 |
4 | 57 | 0 | 0 | 120 | 354 | 0 | 1 | 163 | 1 | 0.6 | 2 | 0 | 2 | 1 |
Note: If you are using Jupyter Notebook to practice this tutorial then there should be no problem to read the CSV file. But if you are a Google fan like me, then you ought to use Google Colab which is the best according to me, for practicing data science, then you must follow some steps in order to load or read the CSV file. So this article helps you to solve this issue (I was the one who wrote this article :0). I personally recommend everybody to go through this article.
This step involves removing irrelevant columns such as cp, fbs, thalach, and many more, and the code is pretty much self-explanatory.
# Dropping unused columns.
to_drop = ['cp',
'fbs',
'restecg',
'thalach',
'exang',
'oldpeak',
'slope',
'thal',
'target',
'ca']
df.drop(to_drop, inplace = True, axis = 1)
df.head(5)
age | sex | trestbps | chol | |
---|---|---|---|---|
0 | 63 | 1 | 145 | 233 |
1 | 37 | 1 | 130 | 250 |
2 | 41 | 0 | 130 | 204 |
3 | 56 | 1 | 120 | 236 |
4 | 57 | 0 | 120 | 354 |
This step involves renaming the column names because many column names are kinda confusing and hard to understand.
# Renaming the column names
new_name = {'age': 'Age',
'sex': 'Sex',
'trestbps': 'Bps',
'chol': 'Cholesterol'
}
df.rename(columns = new_name, inplace = True)
df.head()
Age | Sex | Bps | Cholesterol | |
---|---|---|---|---|
0 | 63 | 1 | 145 | 233 |
1 | 37 | 1 | 130 | 250 |
2 | 41 | 0 | 130 | 204 |
3 | 56 | 1 | 120 | 236 |
4 | 57 | 0 | 120 | 354 |
This step involves replacing the incomplete values or making the values more readable, such as in here the Sex field consists of the values 1 and 0 being 1 as Male and 0 as Female, but it often seems ambiguous for the third person, so changing the value to an understandable one is a good idea.
# Replacing the values in the row
replace_values = {0: 'F', 1: 'M'}
df = df.replace({"Sex": replace_values})
df.head()
Age | Sex | Bps | Cholesterol | |
---|---|---|---|---|
0 | 63 | M | 145 | 233 |
1 | 37 | M | 130 | 250 |
2 | 41 | F | 130 | 204 |
3 | 56 | M | 120 | 236 |
4 | 57 | F | 120 | 354 |
So the above is the overall simple data cleaning process obviously this is not the actual cleaning process at an industry level, but this is a good start, so let’s start from small and then go for huge data-sets which then involves more cleaning process. This was just to give an idea as to how the process of data cleaning looks like in a beginners perspective. Thank you guys for spending your time reading my article, stay tuned for more updates. Let me know what is your opinion about this tutorial in the comment section below. Also if you have any doubts regarding the code, comment section is all yours. Moreover I am an author in towardsdatascience.com and you can find this article there. Have a nice day.