We examine economic and financial time series where Holt-Winters is used to forecast one-year ahead. Daily data for bonds, equity, and gold is then analyzed.
Our focus is on geometric mean returns since they optimally express mean-variance under logarithmic utility. We shall cover portfolio optimization in another notebook.
[ ] TODO: use sympy to symbolically derive geometric mean return from the moments of an asset's return distribution. Our function georet() gives a numerical approximation.
Dependencies:
CHANGE LOG
2016-01-05 MAJOR REWRITE: use pattern from monthly and daily series
for new functions groupget, grouppc, groupgeoret.
Forecast print out replaced by preservable groupholtf.
Dictionary comprehension clarifies code.
2016-01-03 Fix issue #2 with v4 and p6 upgrades.
2015-05-26 Code revision using template v14.12.21.
2014-10-11 Code review. Template 2014-09-28.
2014-09-01 First version.
from fecon235.fecon235 import *
# PREAMBLE-p6.15.1223 :: Settings and system details
from __future__ import absolute_import, print_function
system.specs()
pwd = system.getpwd() # present working directory as variable.
print(" :: $pwd:", pwd)
# If a module is modified, automatically reload it:
%load_ext autoreload
%autoreload 2
# Use 0 to disable this feature.
# Notebook DISPLAY options:
# Represent pandas DataFrames as text; not HTML representation:
import pandas as pd
pd.set_option( 'display.notebook_repr_html', False )
# Beware, for MATH display, use %%latex, NOT the following:
# from IPython.display import Math
# from IPython.display import Latex
from IPython.display import HTML # useful for snippets
# e.g. HTML('<iframe src=http://en.mobile.wikipedia.org/?useformat=mobile width=700 height=350></iframe>')
from IPython.display import Image
# e.g. Image(filename='holt-winters-equations.png', embed=True) # url= also works
from IPython.display import YouTubeVideo
# e.g. YouTubeVideo('1j_HxD4iLn8', start='43', width=600, height=400)
from IPython.core import page
get_ipython().set_hook('show_in_pager', page.as_hook(page.display_page), 0)
# Or equivalently in config file: "InteractiveShell.display_page = True",
# which will display results in secondary notebook pager frame in a cell.
# Generate PLOTS inside notebook, "inline" generates static png:
%matplotlib inline
# "notebook" argument allows interactive zoom and resize.
:: Python 2.7.10 :: IPython 4.0.0 :: jupyter 1.0.0 :: notebook 4.0.6 :: matplotlib 1.4.3 :: numpy 1.10.1 :: pandas 0.17.1 :: pandas_datareader 0.2.0 :: Repository: fecon235 v4.15.1230 develop :: Timestamp: 2016-01-06, 17:06:06 UTC :: $pwd: /media/yaya/virt15h/virt/dbx/Dropbox/ipy/fecon235/nb
We retrieve the following data of monthly frequency: (aggregated) inflation, bonds (zero coupon equivalent of 10-y Treasury), equities (S&P 500), and gold (London PM fix) -- all denominated in US dollars -- then lastly, the real trade-weighted USD index (Federal Reserve) and US home prices (per Case-Shiller). The details for each series is given in their respective notebooks. If the available data has daily frequency, we use the pandas method called "resampling" to induce monthly data (enter "monthly??" in an input cell for more details).
ATTENTION: The inclusion of home prices, unfortunately, will create a 3-month lag, due to their release cycle. Since this is a comparative study, the rest of the data will appear somewhat stale, but this section is intended for long-term trends. Second half of this notebook will examine more responsive daily data.
# Specify monthly series of interest as a dictionary:
msdic = {'Infl' : m4infl, 'Zero10' : m4zero10, 'SPX' : m4spx,
'XAU' : m4xau, 'USD' : m4usdrtb, 'Homes' : m4homepx }
# Download data into a dataframe:
msdf = groupget( msdic )
# "groupget??" at input cell gives function details.
:: Case-Shiller prepend successfully goes back to 1987. :: S&P 500 prepend successfully goes back to 1957.
After downloading the level series, we compute the YoY percentage change for each series. This will be the a trailing 12-month statistic, thus it is overlapping.
# Construct the mega YoY dataframe:
mega = grouppc( msdf, freq=12 )
# Define start time as t0
t0 = '1988'
# We can easily rerun the rest of this notebook
# by specifying another start time, then: Cell > Run All Below
# Slice the data:
stats( mega[t0:] )
Homes Infl SPX USD XAU Zero10 count 334.000000 334.000000 334.000000 334.000000 334.000000 334.000000 mean 3.905883 2.397870 8.708560 -0.034278 4.657274 2.247932 std 8.002381 1.015202 16.260123 5.224677 15.495695 7.390370 min -18.906156 -0.183948 -42.349361 -10.574015 -27.752361 -18.521264 25% -1.069951 1.678485 2.316010 -3.627323 -6.904147 -3.085161 50% 4.490362 2.349958 10.596906 -0.157484 1.745723 2.892972 75% 10.605588 2.847579 19.535143 2.929578 13.956456 7.705685 max 17.077118 5.243597 52.051354 14.502797 60.357143 20.492399 :: Index on min: Homes 2009-01-01 Infl 2009-07-01 SPX 2009-03-01 USD 2008-03-01 XAU 2013-12-01 Zero10 1994-10-01 dtype: datetime64[ns] :: Index on max: Homes 2004-07-01 Infl 1990-10-01 SPX 2010-03-01 USD 2009-03-01 XAU 2006-05-01 Zero10 1996-01-01 dtype: datetime64[ns] :: Head: Homes Infl SPX USD XAU Zero10 T 1988-01-01 12.139461 3.984319 -6.207571 -9.836868 17.042392 -12.661625 1988-02-01 11.661442 3.875810 -8.186030 -8.633477 9.982617 -8.068941 1988-03-01 11.345646 3.923987 -8.156954 -9.393214 8.841782 -9.248410 1988-04-01 10.974485 3.992633 -8.960111 -8.283502 2.878355 -5.477528 1988-05-01 10.543908 4.017451 -11.871026 -7.507266 -1.578661 -4.487925 1988-06-01 10.559567 4.128604 -10.689462 -7.401580 0.022137 -4.738070 1988-07-01 10.756853 4.275829 -12.487441 -6.469468 -3.115990 -5.671538 :: Tail: Homes Infl SPX USD XAU Zero10 T 2015-04-01 4.833607 0.790537 12.280293 10.154619 -7.736721 7.202302 2015-05-01 4.844291 0.815938 12.167416 9.815597 -7.530004 3.148595 2015-06-01 4.841549 0.893118 7.790849 10.680445 -7.225832 2.011656 2015-07-01 4.969711 0.891452 6.329319 12.875617 -12.641221 1.966971 2015-08-01 5.097901 0.907884 6.557453 14.025825 -13.606178 2.058782 2015-09-01 5.391610 0.842386 -2.590674 13.156224 -9.018853 3.285862 2015-10-01 5.565592 0.879616 3.866075 10.834236 -5.184130 2.195704 :: Correlation matrix: Homes Infl SPX USD XAU Zero10 Homes 1.000000 0.005092 0.247100 0.043519 -0.245234 -0.294438 Infl 0.005092 1.000000 -0.008046 -0.357532 -0.047562 -0.066079 SPX 0.247100 -0.008046 1.000000 -0.054522 -0.207546 -0.136715 USD 0.043519 -0.357532 -0.054522 1.000000 -0.523893 0.207311 XAU -0.245234 -0.047562 -0.207546 -0.523893 1.000000 -0.002806 Zero10 -0.294438 -0.066079 -0.136715 0.207311 -0.002806 1.000000
There is not much correlation among our assets, except a mild negative between gold XAU and USD. (2015-05-26 at -0.51)
The boxplot gives us an idea of the range of annual returns, and their persistence due to overlap. Thus trends can be easily discerned.
It is also a visual aid for the geometric mean returns which is most significant as investment metric.
As usual, the red line plots the median, but the red dot represents the latest point.
# Overlapping YoY percentage change, recently:
boxplot(mega[t0:], 'Assets YoYm')
# where the red dot represents the latest point.
:: Finished: boxplot-Assets_YoYm.png
Red dot outside the mid-range box alerts us to unusual conditions. Attention should also be paid to the extreme value "slash" marks (where outliers are also revealed).
David E. Shaw, famous for his proprietary hedge fund, remarked that one of the most important equations in finance is the penalization of arithmetic mean by one-half of variance:
$ g = \mu - (\sigma^2 / 2) $
which turns out to be our geometric mean return. It is an approximation, by the way, but good enough to maximize, instead of considering intricate mean-variance trade-off. We find it useful also as a metric for economic variables.
The source code shows us that georet() first gives us the geometric mean return, followed by the arithmetic mean return and volatility, then finally, the yearly frequency used -- in list format.
# How are we computing geometric mean returns?
# Just add "?" or "??" to variables, procedures, etc.
# to find out the details, e.g.
georet??
Signature: georet(dfx, yearly=256) Source: def georet( dfx, yearly=256 ): '''Compute geometric mean return in a summary list.''' # yearly refers to frequency, e.g. 256 for daily trading days, # 12 for monthly, # 4 for quarterly. #-alt dflg = np.log( dfx ) #-alt dfpc = dflg.diff( periods=1 ) dfpc = dfx.pct_change( periods=1 ) # ^instead of first difference of logged data, # gives slightly higher arithmetic means. mean = dfpc.mean().values.tolist()[0] * yearly vari = dfpc.var().values.tolist()[0] * yearly # ^summary statistics methods, see # McKinney, p.139, Table 5-10. geor = mean - (0.5*vari) # ^arithmetic mean return penalized by risk, # optimal choice under log utility. lst = [ geor, mean, vari ** 0.5 ] # ^^^^^^i.e. std sigma, or volatility. lst = [ round(i*100, 2) for i in lst ] # ^[ geor, mean, volatility ] in readable % form. return lst + [ yearly ] File: ~/Dropbox/ipy/fecon235/lib/yi_1tools.py Type: function
# Geometric mean returns, non-overlapping, annualized:
groupgeoret( msdf[t0:], yearly=12 )
# Note that we applied groupgeoret to msdf, not mega.
# Generally georet requires price levels.
# groupgeoret is just georet for group dataframes.
{'Homes': [3.39, 3.42, 2.53, 12], 'Infl': [2.32, 2.32, 0.5, 12], 'SPX': [7.58, 8.33, 12.25, 12], 'USD': [0.19, 0.28, 4.13, 12], 'XAU': [3.2, 4.05, 13.05, 12], 'Zero10': [2.07, 2.33, 7.15, 12]}
2014-09-01, georet since 2010
2014-10-11, georet since 2004
2014-10-12, georet since 1988
2015-05-27, georet since 1988
2016-01-03, georet since 1988
We forecast one-year ahead using the monthly data. Note that the most current infl level is rebased to 1, thus 1.02 would signify 2% increase.
# These 12-periods ahead forecasts use default alpha and beta values
# found to be optimal for a fixed Kalman filter.
groupholtf( msdf, h=12 )
Homes Infl SPX USD XAU Zero10 0 216295.322178 0.999125 2021.400000 97.185000 1165.050000 83.183904 1 216719.918400 0.999473 2067.207192 99.116894 1126.538411 83.140498 2 217393.597328 1.000433 2063.544333 99.945673 1117.644357 83.208857 3 218067.276256 1.001392 2059.881474 100.774452 1108.750303 83.277216 4 218740.955184 1.002352 2056.218615 101.603231 1099.856249 83.345574 5 219414.634112 1.003312 2052.555756 102.432010 1090.962195 83.413933 6 220088.313040 1.004272 2048.892897 103.260789 1082.068141 83.482292 7 220761.991968 1.005232 2045.230038 104.089568 1073.174088 83.550651 8 221435.670896 1.006192 2041.567179 104.918347 1064.280034 83.619010 9 222109.349824 1.007152 2037.904320 105.747125 1055.385980 83.687369 10 222783.028752 1.008112 2034.241461 106.575904 1046.491926 83.755727 11 223456.707680 1.009072 2030.578602 107.404683 1037.597872 83.824086 12 224130.386608 1.010032 2026.915743 108.233462 1028.703819 83.892445
Changing Holt-Winters alpha from 0.20 to 0.10 varies the forecast only slightly. The important parameter is beta to capture trend effects. Currently we shall rely on default Holt-Winters settings for robustness.
2014-09-01, Twelve-month Forecasts given data through 2014-07-01:
2014-10-11, Twelve-month Forecasts given ten-year data, robust HW:
2015-05-28, Twelve-month Forecasts given data through 2015-03-01, robust HW:
2016-01-03, Twelve-month Forecasts given data through 2015-10-01, robust HW:
We examine bonds (zero coupon equivalent of 10-y Treasury), equities (SPX), gold (XAU), EURUSD, and USDJPY at higher frequency (daily) for the most recent developments.
Inflation, real trade-weighted USD index, and US home price data have a slow monthly release schedule. And for home price data, there is a three month lag.
# Specify daily series of interest as a dictionary
# where key is name, and value is its data code:
dsdic = { 'Zero10' : d4zero10, 'SPX' : d4spx, 'XAU' : d4xau,
'EURUSD' : d4eurusd, 'USDJPY' : d4usdjpy }
# Download data into a dataframe:
dsdf = groupget( dsdic )
:: S&P 500 prepend successfully goes back to 1957.
# Construct the dega YoY percent dataframe:
dega = grouppc( dsdf, freq=256 )
# ^for daily data
# Set the start date for daily series:
u0 = '2010-01-01'
# Plot overlapping percentage changes:
boxplot( dega[u0:], 'Assets YoYd' )
# Note that the "last" timestamp will be more
# recent than for the monthly series.
:: Finished: boxplot-Assets_YoYd.png
Although monthly data is more suitable for making long-term forecasts, daily data is much more sensitive to immediate market perturbations.
2016-01-03 Good example of the foregoing remark is the reaction in the overall market due to the first Fed rate hike in almost a decade on 2015-12-16. ZIRP, zero interest rate program, has been terminated, along with US quantitative easing, thus asset prices must adjust to financing constraints. Note how equities and gold are now below their mid-range boxes.
stats(dega[u0:])
EURUSD SPX USDJPY XAU Zero10 count 1565.000000 1565.000000 1565.000000 1565.000000 1565.000000 mean -3.239333 14.108931 4.967449 4.568358 1.861880 std 9.126193 10.118387 11.876045 18.700101 6.470399 min -23.984762 -6.776581 -14.310589 -29.210332 -13.248728 25% -10.613876 7.172626 -6.285016 -9.629768 -1.231181 50% -2.304728 13.200272 2.984473 -0.627657 2.419923 75% 3.984704 19.505195 15.422171 23.593619 5.626088 max 21.056554 65.300874 30.357599 52.361809 18.075299 :: Index on min: EURUSD 2015-03-11 SPX 2015-08-25 USDJPY 2011-03-17 XAU 2013-12-26 Zero10 2010-01-07 dtype: datetime64[ns] :: Index on max: EURUSD 2011-06-06 SPX 2010-03-02 USDJPY 2013-05-28 XAU 2011-09-06 Zero10 2012-02-01 dtype: datetime64[ns] :: Head: EURUSD SPX USDJPY XAU Zero10 T 2010-01-01 4.506344 22.574830 2.117389 27.081507 -11.463243 2010-01-04 6.846980 27.252204 2.548476 32.369431 -11.776234 2010-01-05 7.904398 30.595454 2.878992 35.822249 -11.858700 2010-01-06 9.137748 30.437376 3.212493 36.721113 -12.554147 2010-01-07 8.653408 35.492867 4.513889 37.583688 -13.248728 2010-01-08 9.645639 35.702942 3.471370 39.104938 -13.173018 2010-01-11 9.755361 34.919776 1.783143 38.290855 -12.321456 :: Tail: EURUSD SPX USDJPY XAU Zero10 T 2015-12-23 -10.670281 -0.771985 1.340707 -11.422056 -0.618486 2015-12-24 -9.470292 0.101511 0.392157 -11.422056 -0.706616 2015-12-25 -9.470292 0.101511 0.392157 -11.422056 -0.706616 2015-12-28 -8.589263 -0.082596 0.083195 -8.852389 -1.058205 2015-12-29 -8.407451 2.859575 0.668673 -10.825000 -2.451747 2015-12-30 -8.579088 3.033541 1.978691 -12.414790 -2.969830 2015-12-31 -8.130288 0.890468 0.627510 -12.432879 -2.711703 :: Correlation matrix: EURUSD SPX USDJPY XAU Zero10 EURUSD 1.000000 0.471964 -0.163779 0.158736 -0.524794 SPX 0.471964 1.000000 0.090843 -0.052154 -0.687276 USDJPY -0.163779 0.090843 1.000000 -0.859083 -0.400329 XAU 0.158736 -0.052154 -0.859083 1.000000 0.409829 Zero10 -0.524794 -0.687276 -0.400329 0.409829 1.000000
2015-05-29, Suprisingly, very little correlation between EURUSD and USDJPY: -6%. Gold appears more correlated with USDJPY at -87% than EURUSD at +6%
2016-01-05, Given the latest Fed hike, the correlation to watch is between equities and bonds (-0.69 SPX and Zero10).
# What are the latest daily prices?
tail( dsdf )
EURUSD SPX USDJPY XAU Zero10 T 2015-12-23 1.0875 2064.29 120.94 1068.25 81.685673 2015-12-24 1.0955 2060.99 120.32 1068.25 81.830579 2015-12-25 1.0955 2060.99 120.32 1068.25 81.830579 2015-12-28 1.0983 2056.50 120.30 1068.25 81.903134 2015-12-29 1.0916 2078.36 120.44 1070.10 81.324593 2015-12-30 1.0912 2063.36 120.60 1060.00 81.396674 2015-12-31 1.0859 2043.94 120.27 1060.00 81.685673
# Geometric mean returns, non-overlapping, annualized:
groupgeoret( dsdf[u0:], yearly=256 )
{'EURUSD': [-4.54, -4.08, 9.64, 256], 'SPX': [9.92, 11.16, 15.76, 256], 'USDJPY': [4.19, 4.63, 9.3, 256], 'XAU': [-0.41, 1.08, 17.27, 256], 'Zero10': [2.28, 2.56, 7.48, 256]}
2014-10-11, Really near-term picture is too bright for SPX while XAU looks dark. Sell stocks, and start to accumulate gold.
2015-05-28, XAU georet changed from 2.6% to 1.6%. Zero10 monthly forecast is basically unchanged. Real rate is what matters for gold. USD stronger by 4.8% against both the EUR and JPY.
2016-01-03, XAU georet changed from 1.6% to -0.41%, commodities including oil going through a bear market. Bonds have not sold off despite 2015-12-16 Fed rate hike, probably due to world appetite for USD which is stronger by about 4.3% against EUR and JPY. SPX looks vulnerable given the past maxims about rate hikes, but the Fed is actually still very accomodative.
[ ] TODO: notebook on r* the so-called natural interest rate.