What is Autocorrelation? ¶

Recall that a classical linear regression model $$ Y_t = \beta_1 +\beta_2 X_t +u_t $$

has an assumption $$ E\left(u_{i} u_{j}\right)=0 \quad i \neq j $$ i.e. disturbance terms are independent of each other. However, in many real situations, this assumption will not hold, especially in time series data. Many economic or financial time series data has a 'momentum', for instance, if we see a boost in consumer confidence in this month, it is more likely continue the momentum next month unless something unexpected happens.

However if this assumption is violated, we have autocorrelation issue $$ E\left(u_{i} u_{j}\right)\neq 0 \quad i \neq j $$ There are many reasons why autocorrelation exists, but for now we don't care about the exact reason.

If we say a series is positively autocorrelated, it means correlation between successive values are positive, vice verse we call it negatively autorrelated. In financial context, we say that positive autocorrelation is trend following(momentum), the negative autocorrelation is mean reverting.

For instance, weekly return of Microsoft (MSFT) has a negative autocorrelation, which means the possibility of mean-reverting trading strategy.

What about interest rate, let's try with 10 year treasury yield.

We can see that interest rate has considerably high negative correlation at the annual change rate, but not the daily change rate. alpha is significance level such that only $5\%$ of chance an autocorrelation falls outside of blue area.

Does White Noise Have Autocorrelation?¶

Stocks price are random walk, but the returns are usually white noises, so we can't use any models to predict returns of stock prices.

Seasonality And Autocorrelation ¶

If you have seasonal data, first order difference won't work, have to use the fourth order difference.

AR And MA Processes ¶

If the disturbance term is generated by the process $$ u_{t}=\rho u_{t-1}+\varepsilon_{t} \quad-1<\rho<1 $$ we call it a First-Order Autoregression Process, denoted $AR(1)$. And innovation $\varepsilon_{t}$ satisfies standard features of OLS disturbance term as below, and $\rho$ is coefficient of autocorrelation at lag $1$ $$ \begin{aligned} E\left(\varepsilon_{t}\right) &=0 \\ \operatorname{Var}\left(\varepsilon_{t}\right) &=\sigma_{\varepsilon}^{2} \\ \operatorname{Cov}\left(\varepsilon_{t}, \varepsilon_{t+s}\right) &=0 \quad s \neq 0 \end{aligned} $$ If $|\rho|<1$ we say that $AR(1)$ process is stationary, we'll get back to this topic in more details.

The main alternative is moving average model which is a weighted sum of current and previous values of $\varepsilon$'s, for instance $MA(2)$ is $$ u_t = \lambda_0 \varepsilon_t+ \lambda_1 \varepsilon_{t-1}+ \lambda_2 \varepsilon_{t-2} $$

Causes Of Autocorrelation¶

Model Misspecification¶

Omitted Variable Bias¶

Be aware low $dw$ test might signal the omission of variable, but not vice versa, we can demonstrate with Monte Carlo simulation.

OLS Estimation With Presence Of Autocorrelation ¶

As in the case of heteroscedasticity, in the presence of autocorrelation the OLS estimatorsare still linear unbiased as well as consistent and asymptotically normally distributed, but they are no longer efficient (i.e., minimum variance).

First use a $AR(1)$ model to generate an autocorrelated series as our disturbance term $$ u_{t}=\rho u_{t-1}+\varepsilon_{t} $$ choose any $\rho$ fits your preference, here I use $.98$

Then assume $X$ fixed at $1, 2, 3,..., T$ and define a true relationship as $$ Y_t = 2 + 3 X_t + u_t $$

You would be surprised to see that true relationship (Population Regression Function) deviates much more than fitted line, it is exactly due to autocorrelation, as $\rho$ getting smaller, the generated dots will be closer to PRF, therefore fitted line will be closer to PRF as well.

The point is that with presence of autocorrelation, the estimation results will vary wildly from true relationship, hence all hypothesis tests are invalid too. This happens very common in time series estimation.

Detection of Autocorrelation ¶

Graphical Method ¶

Though the non-autocorrelation assumption relates to disturbance term $u_t$, which are unobservable, we can instead visualize the residuals $e_t$. Very likely residuals can provide useful information when demonstrating a visual examination.

Here is a simulated residuals from example above, though graphical method is subjective and qualitative, you can still draw to a rough conclusion if you see an patter presence in residuals.

Durbin-Watson Test ¶

The standard test for first order autocorrelation is Durbin-Watson Test, it is one of standard output of OLS estimation. It is calculated by the formula $$ dw=\frac{\sum_{t=2}^{T}\left(e_{t}-e_{t-1}\right)^{2}}{\sum_{t=1}^{T} e_{t}^{2}} $$

where $e_t$ is residual at time $t$.

As an example, we can reproduce the OLS estimation of disposable income and house price in China.

Or we can write our own Durbin-Watson function to verify the results

But how to interpret this number? Some mathematical manipulations of Durbin-Watson's statistic will bring some insight $$ dw=\frac{\sum_{t=2}^{T}\left(e_{t}-e_{t-1}\right)^{2}}{\sum_{t=1}^{T} e_{t}^{2}} = \frac{\sum_{t=2}^{T}\left(e^2_{t}-2e_te_{t-1}+e^2_{t-1}\right)}{\sum_{t=1}^{T} e_{t}^{2}} = \frac{\sum_{t=1}^Te_t^2}{\sum_{t=1}^Te_t^2}+\frac{\sum_{t=1}^Te_{t-1}^2}{\sum_{t=1}^Te_t^2}-2\frac{\sum_{t=1}^Te_te_{t-1}}{\sum_{t=1}^Te_t^2} $$

If $T\rightarrow \infty$, the second term approaches $1$ infinitely and the third term $\frac{\sum_{t=1}^Te_te_{t-1}}{\sum_{t=1}^Te_t^2}$ equals $\rho$, because it is a simple OLS estimator of $\rho$.

Therefore $dw \rightarrow 2-2\rho$ given $T\rightarrow \infty$.

If no autocorrelation, $\rho=0$, $dw$ should be $2$, if $\rho > .5$, i.e. strong positive autocorrelation, $dw$ will be close to $0$, or if $\rho>-.5$, i.e. strong negative autocorrelation, $dw$ will be close to $4$.

Because $dw$ statistic doesn't have critical value (due to some technical reason, critical value can't be calculated.) However, it is possible to know the upper and lower limit of $dw$, go search Durbin Watson table on Google and click any of image that shows on top, or click here.

In the data of flat price, number of observations is $25$ and number of independent variables is $1$, here are upper and lower limit from the table. $$ d_L = 1.055\\ d_U = 1.210 $$ Our result $dw=0.9255$ is lower than lower limit, which suggests a rejection of null hypothesis $\rho=0$, why? Because it's too far away from $2$, if $dw$ is larger than upper limit, we fail to reject null hypothesis. If the $dw$ drops between lower and upper limit, just forget about it, no decisive conclusion.

However, keep in mind that autocorrelation can also be attributed to model specification errors and we almost only care about autocorrelation issue in time series rather than cross sectional data.

Here is an example of exchange rate $EUR/USD$.

The $dw$ statistic is literally $0$, which means $\rho=1$, this shouldn't be a shocking result. Most of time sereis of assets price will show a strong first degree autocorrelation, this is why we like trading the trend or momentum.

However, $dw$ test is mostly for testing residuals, here's another example that demonstrating highly autocorrelated residuals.

The drawback of $dw$ test is that it assumes nonstachastic regressors (or independent variables we call in previous contexts), i.e. the regressors are fixed in repeated sampling. Then this drawback would invalidate the power of $dw$ test in time series modelling.

Breusch–Godfrey Test ¶

The Breusch–Godfrey Test is more general than Durbin-Watson that it allows for higher autoregression schemes and higher order of moving averages of white noises.

Suppose disturbance terms are generated by an $AR(3)$ process $$ u_{t}=\rho_1 u_{t-1}+\rho_2 u_{t-2}+\rho_3 u_{t-3}+\varepsilon_{t} $$ and choose whatever $\rho$'s you like. $\varepsilon_t$ follows classical linear regression assumption.

Once you have residual, run auxiliary regression $$ \hat{u}_{t}=\alpha_{1}+\alpha_{2} X_{t}+\hat{\rho}_{1} \hat{u}_{t-1}+\hat{\rho}_{2} \hat{u}_{t-2}+\cdots+\hat{\rho}_{p} \hat{u}_{t-p}+\varepsilon_{t} $$

Use the coefficient of determinant to construct the statistic $$ (n-p) R^{2} \sim \chi_{p}^{2} $$ where $p$ is the number of lags, $n$ is the number of observations.

Fortunately, statsmodels has Breusch-Godfrey function, the first value is the test statistic and the second is the $p$-value.

Remedies For Autocorrelaton ¶

If we know $\rho$'s, it would be easy to convert to a generalized difference equation form $$ \left(Y_{t}-\rho Y_{t-1}\right)=\beta_{1}(1-\rho)+\beta_{2}\left(X_{t}-\rho X_{t-1}\right)+\varepsilon_{t} $$ such that disturbance term now is $\varepsilon_t$, which satisfied classical OLS assumption.

However, we rarely know true value of $\rho$'s.

If you have no information of $\rho$, the best guess is by check $dw$ test, if $$ dw < R^2 $$ We assume $\rho =1$, then generalized difference equations becomes $$ Y_{t}- Y_{t-1}=\beta_{2}(X_{t}-X_{t-1})+\varepsilon_{t} $$ or $$ \Delta Y_{t}=\Delta \beta_{2}X_{t}+\varepsilon_{t} $$

If you don't want to guess or simply using first order difference, the best choice is to estimate $\rho$ in model $$ e_{t}=\rho \cdot e_{t-1}+v_{t} $$ where $e$'s are residuals.

Because $\rho$'s are estimates, we usually call these procedures the feasible generalized least squares (FGLS).

Instead of using FGLS, we have HAC (heteroscedasticity- and autocorrelation-consistent) standard errors (some textbooks call it Newey-West standard errors).

In small samples, FGLS and HAC are not better than OLS, by small samples, we mean a sample less than $30$ observation.