import addutils.toc ; addutils.toc.js(ipy_notebook=True)
With this tutorial we are going to see some of the statistical and computational tools offered by pandas
.
import datetime
import scipy.io
import numpy as np
import pandas as pd
import bokeh.plotting as bk
from IPython.display import display, HTML
from addutils import css_notebook, side_by_side2
css_notebook()
Given a pandas.Series
the method pct_change
returns a new pandas.Series
object containing percent change over a given number of periods.
s1 = pd.Series(range(10, 18) + np.random.randn(8) / 10)
pct_ch_1d = s1.pct_change() * 100
pct_ch_3d = s1.pct_change(periods=3) * 100
HTML(side_by_side2(s1, pct_ch_1d, pct_ch_3d))
Given two pandas.Series
the method cov
computes covariance between them, excluding missing values.
s1 = pd.util.testing.makeTimeSeries(7)
s2 = s1 + np.random.randn(len(s1)) / 10
HTML(side_by_side2(s1, s2))
s1.cov(s2)
It is also possibile to compute pairwise covariance of a pandas.DataFrame
columns using pandas.DataFrame.cov
method. Here we use the module pandas.util.testing
in order to generate random data easily:
d1 = pd.util.testing.makeTimeDataFrame()
print (d1.head())
print (d1.cov())
pandas.Series.corr
allows to compute correlation between two pandas.Series
. By the method
paramether it's possible to choose between:
s1.corr(s2, method='pearson')
Like we just seen for covariance, it is possibile to call pandas.DataFrame.corr
to obtain pairwise correlation of columns over a pandas.DataFrame
d1.corr()
pandas
provides also a lot of methods for calculating rolling moments.
[n for n in dir(pd) if n.startswith('rolling')]
Let's see some examples:
s3 = pd.Series(np.random.randn(1000), index=pd.date_range('1/1/2000', periods=1000))
s3 = s3.cumsum()
s3_max = s3.rolling(60).max()
s3_mean = s3.rolling(60).mean()
s3_min = s3.rolling(60).min()
data = {'cumsum':s3, 'max':s3_max, 'mean':s3_mean, 'min':s3_min}
df = pd.DataFrame(data)
df.tail()
bk.output_notebook()
fig = bk.figure(x_axis_type = "datetime",
tools="pan,box_zoom,reset", title = 'Rolling Moments',
plot_width=750, plot_height=400)
fig.line(df.index, df['cumsum'], color='cadetblue', legend='Cumulative Sum')
fig.line(df.index, df['max'], color='mediumorchid', legend='Max')
fig.line(df.index, df['min'], color='mediumpurple', legend='Min')
fig.line(df.index, df['mean'], color='navy', legend='Mean')
bk.show(fig)
pandas.Series.cumsum
returns a new pandas.Series
containing the cumulative sum of the given values.
s4 = s3 + np.random.randn(len(s3))
rollc = s3.rolling(window=10).corr(s3)
data2 = {'cumsum':s3, 'similar':s4, 'rolling correlation':rollc}
df2 = pd.DataFrame(data2)
fig = bk.figure(x_axis_type = "datetime", title = 'Rolling Correlation',
plot_width=750, plot_height=400)
fig.line(df2.index, df2['cumsum'], color='cadetblue', legend='Cumulative Sum')
fig.line(df2.index, df2['similar'], color='mediumorchid', legend='Similar')
fig.line(df2.index, df2['rolling correlation'], color='navy', legend='Rolling Corr.')
fig.legend.location = "bottom_right"
bk.show(fig)
AAPL = pd.read_csv('example_data/p03_AAPL.txt', index_col='Date', parse_dates=True)
price = AAPL['Adj Close']
display(price.tail())
pandas.Series.tail
returns the last n rows of a given pandas.Series
.
price['2011-10-03'] / price['2011-3-01'] - 1
returns = price.pct_change()
ret_index = (1 + returns).cumprod()
ret_index[0] = 1
monthly_returns = ret_index.resample('BM').last().pct_change()
fig = bk.figure(x_axis_type = 'datetime', title = 'Monthly Returns',
plot_width=750, plot_height=400)
fig.line(monthly_returns.index, monthly_returns)
bk.show(fig)
Visit www.add-for.com for more tutorials and updates.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.