Python Pandas DataFrame basics.¶

Let us understand the basics of Pandas DataFrame from scratch.¶

alt text

Before getting started let me introduce you Pandas, Pandas is a python library which provided high-performance, easy-to-use data structures such as a series, Data Frame and Panel for data analysis tools for Python programming language. Moreover Pandas Data Frame consists of main components, the data, rows and columns. To use the pandas library and its data structures, all you have to do it to install it and import it. See the documentation of the Pandas library for a better understanding and installing guidance.

Basic operations that can be appilied on a pandas Data Frame are as shown below:¶

Creating a Data Frame.
Performing operations on Rows and Columns.
Data Selection, addition, deletion.
Working with missing data.
Renaming the Columns or Indices of a DataFrame.

1. Creating a pandas DataFrame¶

The pandas data frame can be created by loading the data from the external, existing storage like a database, SQL or CSV files. But the pandas Data Frame can also be created from the lists, dictionary, etc. One of the ways to create a pandas data frame is shown below:

In [0]:

# import the pandas library
import pandas as pd
# Dictionary of key pair values called data
data = {'Name':['Ashika', 'Tanu', 'Ashwin', 'Mohit', 'Sourabh'],
        'Age': [24, 23, 22, 19, 10]}
data

Out[0]:

{'Age': [24, 23, 22, 19, 10],
 'Name': ['Ashika', 'Tanu', 'Ashwin', 'Mohit', 'Sourabh']}

In [0]:

# Calling the pandas data frame method by passing the dictionary (data) as a parameter
df = pd.DataFrame(data)
df

Out[0]:

	Name	Age
0	Ashika	24
1	Tanu	23
2	Ashwin	22
3	Mohit	19
4	Sourabh	10

2. Performing operations on Rows and Columns.¶

Data Frame is a two-dimensional data structure, data is stored in rows and columns. Below we can perform some operations on Rows and Columns.

Selecting a Column

In order to select a particular column, all we can do is just call the name of the column inside the data frame.

In [0]:

# import the pandas library
import pandas as pd
# Dictionary of key pair values called data
data = {'Name':['Ashika', 'Tanu', 'Ashwin', 'Mohit', 'Sourabh'],
        'Age': [24, 23, 22, 19, 10]}
data

Out[0]:

{'Age': [24, 23, 22, 19, 10],
 'Name': ['Ashika', 'Tanu', 'Ashwin', 'Mohit', 'Sourabh']}

In [0]:

# Calling the pandas data frame method by passing the dictionary (data) as a parameter
df = pd.DataFrame(data)
# Selecting column
df[['Name']]

Out[0]:

	Name
0	Ashika
1	Tanu
2	Ashwin
3	Mohit
4	Sourabh

Selecting a Row

Pandas Data Frame provides a method called "loc" which is used to retrive rows from the data frame. Also rows can also be selected by using the "iloc" as an function.

In [0]:

# import the pandas library
import pandas as pd
# Dictionary of key pair values called data
data = {'Name':['Ashika', 'Tanu', 'Ashwin', 'Mohit', 'Sourabh'],
        'Age': [24, 23, 22, 19, 10]}
data

Out[0]:

{'Age': [24, 23, 22, 19, 10],
 'Name': ['Ashika', 'Tanu', 'Ashwin', 'Mohit', 'Sourabh']}

In [0]:

# Calling the pandas data frame method by passing the dictionary (data) as a parameter
df = pd.DataFrame(data)
# Selecting a row
row = df.loc[1]
row

Out[0]:

Name    Tanu
Age       23
Name: 1, dtype: object

To select a particular column, all we can do is just call the name of the column inside the data frame. As seen above to work with the “loc” method you have to pass the index of the data frame as a parameter. The loc method accepts only integers as a parameter. So in the above example, I wanted to access “Tanu” row, so I passed the index as 1 as a parameter. Now theres a quick assignment for you guys, use the "iloc" method and tell me the result.

3. Data Selection, addition, deletion.¶

You can treat a DataFrame semantically like a dict of like-indexed Series objects. Getting, setting, and deleting columns works with the same syntax as the analogous dict operations:

In [0]:

# import the pandas library
import pandas as pd
# Dictionary of key pair values called data
data = {'Name':['Ashika', 'Tanu', 'Ashwin', 'Mohit', 'Sourabh'],
        'Age': [24, 23, 22, 19, 10]}
data

Out[0]:

{'Age': [24, 23, 22, 19, 10],
 'Name': ['Ashika', 'Tanu', 'Ashwin', 'Mohit', 'Sourabh']}

In [0]:

# Calling the pandas data frame method by passing the dictionary (data) as a parameter
df = pd.DataFrame(data)
# Selecting the data from the column
df['Age']

Out[0]:

0    24
1    23
2    22
3    19
4    10
Name: Age, dtype: int64

Columns can be deleted like with a dict:

In [0]:

del df['Age']
df

Out[0]:

	Name
0	Ashika
1	Tanu
2	Ashwin
3	Mohit
4	Sourabh

Data can be added by using the insert function. The insert function is available to insert at a particular location in the columns:

In [0]:

df.insert(1, 'name', df['Name'])
df

Out[0]:

	Name	name
0	Ashika	Ashika
1	Tanu	Tanu
2	Ashwin	Ashwin
3	Mohit	Mohit
4	Sourabh	Sourabh

4. Working with missing data.¶

Missing data occur a lot of times when we are accessing big data sets. It occurs often like NaN (Not a number). In order to fill those values we can use "isnull()" method. This method checks whether a null value is present in a data frame or not.

Checking for the missing values.

In [0]:

# importing both pandas and numpy libraries
import pandas as pd
import numpy as np

# Dictionary of key pair values called data
data ={'Name':['Tanu', np.nan],
       'Age': [23, np.nan]}
data

Out[0]:

{'Age': [23, nan], 'Name': ['Tanu', nan]}

In [0]:

df = pd.DataFrame(data)
df

Out[0]:

	Name	Age
0	Tanu	23.0
1	NaN	NaN

In [0]:

# using the isnull() function
df.isnull()

Out[0]:

	Name	Age
0	False	False
1	True	True

Now we have found the missing values, next task is to fill those values with 0 this can be done as shown below:

In [0]:

df.fillna(0)

Out[0]:

	Name	Age
0	Tanu	23.0
1	0	0.0

5. Renaming the Columns or Indices of a DataFrame..¶

To give the columns or the index values of your data frame a different value, it’s best to use the .rename() method. Purposefully I have changed the column name to give a better insight

In [0]:

# import the pandas library
import pandas as pd
# Dictionary of key pair values called data
data = {'NAMe':['Ashika', 'Tanu', 'Ashwin', 'Mohit', 'Sourabh'],
        'AGe': [24, 23, 22, 19, 10]}
data

Out[0]:

{'AGe': [24, 23, 22, 19, 10],
 'NAMe': ['Ashika', 'Tanu', 'Ashwin', 'Mohit', 'Sourabh']}

In [0]:

# Calling the pandas data frame method by passing the dictionary (data) as a parameter
df = pd.DataFrame(data)
df

Out[0]:

	NAMe	AGe
0	Ashika	24
1	Tanu	23
2	Ashwin	22
3	Mohit	19
4	Sourabh	10

In [0]:

newcols = {
            'NAMe': 'Name',
            'AGe': 'Age'
          }
# Use `rename()` to rename your columns
df.rename(columns=newcols, inplace=True)
df

Out[0]:

	Name	Age
0	Ashika	24
1	Tanu	23
2	Ashwin	22
3	Mohit	19
4	Sourabh	10

In [0]:

# The values of new index
newindex = {
            0: 'a',
            1: 'b',
            2: 'c',
            3: 'd',
            4: 'e'
}
# Rename your index
df.rename(index=newindex)

Out[0]:

	Name	Age
a	Ashika	24
b	Tanu	23
c	Ashwin	22
d	Mohit	19
e	Sourabh	10