#!/usr/bin/env python
# coding: utf-8

# # **The Euro on Forex**
# ## **Exploring Twenty Years of Volatility**
# ### Very Brief History
# The currency exchange market has been around since coins were first used. At the end of the 1800s the development of the gold standard fixed currency values to a certain amount of gold, allowing for an internationally stable monetary system (1). Eventually the amount of gold was not enough to keep up with the demand for currency. After World War II the Bretton Woods Accord set the US Dollar as the global fixed currency. This created an adjustable rate exchange, but it was still determined by central banks and state actors (2). The US dollar was pegged to gold at the time, and during the 1970s the same liquidity problem (i.e. amount of gold vs currency in circulation) occurred. In August of 1971 President Richard Nixon unilaterally canceled direct international convertibility of the US dollar to gold, inadvertently beginning a regime of free-floating currencies (3).
# 
# ### Today
# The modern foreign exchange market (Forex, FX, or currency market) is a direct broker to dealer global marketplace that determines the market value and exchange rate of currencies all over the world. It is open 24 hours a day, 5 days a week. Currencies are always traded in pairs, and the value expressed is of one currency relative to another. For example, at the end of the day on 11/19/21 the Euro to US dollar rate was 1.13, or $1.13 equals €1. All of the twenty most traded currencies are free-floating, their value is determined by political conditions, economic factors and market psychology.
# 
# Forex is by far the largest financial market in the world, in 2019 having a daily volume of over $6.6 trillion (4). The Euro to US dollar (EURUSD) is the most traded currency pair, taking about 30% of the multi-billion dollar Forex turnover (5).
# 
# ### Methodology
# This study examines the volatility of the value of the Euro to eight other currencies. It uses the 2019 Triennial Central Bank Survey by the Bank for International Settlements to determine four currencies that represent large portions of the market and four currencies that represent small portions of the market. The highly traded currencies are the US dollar, Japanese yen, UK pound, and Australian dollar. The Israeli shekel, Phillipine peso, Malaysian ringgit, and Romanian leu comprise the group of least traded currencies.
# 
# The data set used is based on daily closing values provided by the European Central Bank and compiled by Daria Chemkaeva. It is updated weekly, the version used for this study was downloaded from Kaggle in November 2021.
# 
# ### Summary
# Text with conclusion.
# <br><br>
# 
# ## **Initial Look**
# The first steps are to import the basic libraries, open the file, and take an initial look at the dataset.
# 
# 

# In[1]:


# import libraries and set display options
import pandas as pd
import pprint

# set frame width and center output
from IPython.core.display import display, HTML
display(HTML("""
<style>
.container {width:84% !important;
}
</style>
"""))
# display(HTML("""
# <style>
# .output {
#     display: flex;
#     align-img: center;
#     text-align: left;
# }
# .container {width:70% !important;
# }
# </style>
# """))


# pretty print options
pd.options.display.float_format = '{:20,.4f}'.format
pd.set_option('display.max_rows', None)
pd.set_option('display.max_columns', None)
pd.set_option('display.width', 3000)
pd.set_option('display.colheader_justify', 'center')
pd.set_option('display.precision', 3)

# open file
x_rates = pd.read_csv("euro-daily-hist_1999_2020.csv")

# head, tail, and basic info
# display(x_rates.head())
# display(x_rates.tail())
# display(x_rates.info())


# ## **Initial Look Continued**
# The dataset covers the time period from January 1999 to January 2021. It has 5699 rows and 41 columns. Some of these columns have less than half the number of entries as other columns. In the original data set, cells with missing entries have a hyphen (-). This causes problems with a lot of the functions that will be used. For the time being the hyphens will be converted to the value ```NaN```, which preserves the row of information.
# ## **Data Cleaning**
# The first steps will be:
# * Clean and modify column names
# * Convert to `datetime` data type
# * Sort the values with oldest first and reset the index
# * Clean currency columns
# * Convert exchange rates to floats

# In[2]:


# strip brackets and trailing space from country names
# replace remaining space with underscore
# lower case all column names
x_rates.columns = x_rates.columns.str.replace("[","").str.replace("]","")
x_rates.columns = x_rates.columns.str.rstrip()
x_rates.columns = x_rates.columns.str.replace(" ","_")
x_rates.columns = x_rates.columns.str.lower()

# rename columns
x_rates.rename(columns={"period\\unit:":"date",
                        "chinese_yuan_renminbi":"chinese_yuan",
                        "uk_pound_sterling":"uk_pound"}, inplace=True)

# convert datetime
x_rates["date"] = pd.to_datetime(x_rates["date"])

# resort and reindex
x_rates.sort_values("date", inplace=True)
x_rates.reset_index(drop=True, inplace=True)

# convert hyphens in currency columns to NaN
import numpy as np
x_rates = x_rates.replace("-", np.nan)

# convert exchange rate values to float
x_rates.iloc[:,1:] = x_rates.iloc[:,1:].astype(float)

display(x_rates.head())


# ## **Euro to Dollar**
# A line graph is a great way to look at change over time. The Euro to US dollar is the first trading pair to be examined. This will also continue building a good foundation for developing the project.

# In[3]:


# create a us dollar to euro dataframe and add 30 day rolling mean column
us_dollar = x_rates[["date","us_dollar"]].copy()
us_dollar["rolling_mean_30"] = us_dollar["us_dollar"].rolling(30).mean()

# line graph for dollar to euro exchange rate
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
get_ipython().run_line_magic('matplotlib', 'inline')
plt.style.use('ggplot') # set style for graphs
fig, ax = plt.subplots(figsize=(18,10))
ax.plot(us_dollar["date"], us_dollar["rolling_mean_30"], color="royalblue", linewidth=3)
ax.set_xticks(["2000-01-04 00:00:00", "2005-01-04 00:00:00", "2010-01-04 00:00:00",
               "2015-01-04 00:00:00", "2020-01-04 00:00:00"])
ax.set_xticklabels([2000,2005,2010,2015,2020], fontsize=18, weight="bold")
# ax.set_yticks([0,0.9,1.0,1.1,1.2,1.3,1.4,1.5,1.6])
ax.set_yticklabels([0,0.9,1.0,1.1,1.2,1.3,1.4,1.5,1.6], fontsize=18, weight="bold")
ax.text(0.08, 0.0, "© B McMinn" + " "*181 + "Source: Daria Chemkaeva",
        color = "#f0f0f0", backgroundcolor = "#4d4d4d", fontsize=14,
        transform=fig.transFigure)
fig.suptitle(" "*10 + "Euro to US Dollar Currency Exchange Rates" + " "*10,
             color = "#f0f0f0", backgroundcolor = "#4d4d4d", fontsize=35, weight="bold")
plt.show()


# ## **Euro to Dollar Continued**
# * The line graph shows that the exchange rate between the Euro and US dollar is pretty variable.
# * This is the only graph showing a value for the currency being traded against the Euro. It shows that at it's highest value &euro;1 traded for about \\$1.60. At one point the Euro trades for less than \\$1. However for this study, comparing across currencies and looking at particular regional events makes the actual currency amount less relevant.
# * There are also a lot of gaps in the graph. These represent points in the data frame where information for a particular day is missing. This is the most accurate representation of the data, but it's not very easy to look at. A quick visual check can illustrate this overlap and then compare the original data frame with one where the missing dates are removed.

# In[4]:


# create nan group and get dates
nan_dollar_group = us_dollar[us_dollar["us_dollar"].isnull()]
nan_days = nan_dollar_group["date"]

# create a dollar to euro dataframe with NaN dates removed
US_dollar_nonull = x_rates[x_rates["us_dollar"].notna()].copy()
US_dollar_nonull["rolling_mean_30"] = US_dollar_nonull["us_dollar"].rolling(30).mean()

# line graph showing dollar to euro showing NaN values with and without highlight
fig, (ax1, ax2, ax3) = plt.subplots(nrows=3, ncols=1, figsize=(18,10))
ax1.plot(us_dollar["date"], us_dollar["rolling_mean_30"], color="royalblue", linewidth=5)
# ax1.set(title="Dataset with NaN Values Included")
ax1.set_title("Dataset with NaN Values Included", fontsize=18)
ax2.plot(us_dollar["date"], us_dollar["rolling_mean_30"], color="royalblue", linewidth=5)
ax2.set_title("Dataset with Corresponding Dates in Gold", fontsize=18)
for xc in nan_days:
   ax2.axvline(x=xc, color="gold", linewidth=4)
ax3.plot(US_dollar_nonull["date"], US_dollar_nonull["rolling_mean_30"], color="salmon", linewidth=5)
ax3.set_title("Dataset with NaN Rows Removed", fontsize=18)
for ax in ax1, ax2, ax3:
    ax.set_xticks([])
    ax.set_xticklabels([])
for ax in ax1, ax2, ax3:
    ax.set_yticks([])
ax.text(0.08, 0.0, "© B McMinn" + " "*181 + "Source: Daria Chemkaeva",
        color = "#f0f0f0", backgroundcolor = "#4d4d4d", fontsize=14,
        transform=fig.transFigure)
fig.suptitle(" "*10 + "Euro to US Dollar Currency Exchange Rates" + " "*10,
             color = "#f0f0f0", backgroundcolor = "#4d4d4d", fontsize=35, weight="bold")
plt.show()


# ## **Euro to Dollar Conclusion**
# The second graph shows vertical gold stripes for all dates with missing information. These correspond with the gaps in the original chart. The third graph, with these rows removed, appears to be good match and is easier to look at. This also demonstrates how the rolling mean handles NaN values within a data frame. In either case the shape of the line is identical where the information exists, therefor both are similarly accurate representations of the available data. This would look very different if the program managed days with missing data as 0. All further currency data frames will be handled the same.
# 
# ## **Euro & Traded Pairs**
# ### Euro to Highly Traded Currencies
# The first set of line graphs represent the exchange rates for the Euro to the US dollar, Japanese yen, UK pound, and Australian dollar. These four currencies, in addition to the Euro, are the top five currencies in the Forex market. All five together comprise over 75% of daily trades (4).
# ### Euro to Least Traded Currencies
# The second set of graphs represent the exchange rates for the Euro to the Israeli shekel, Philippine peso, Malaysian ringgit, and Romanian leu. These four currencies have low trade volumes, they comprise less than 0.4% of daily trades (4).

# In[5]:


# import colormap
from matplotlib.pyplot import cm
colors = iter([plt.cm.Set3(i) for i in range(20)])

# create a list of data frame names
# not sure this is neccessary but trying df = df_list.index(currency) in the framer function doesn't seem to work
# create a list of data frames for each currency with log rate of the exchange rate, 30 day rolling mean, and year
# col_names = []
df_dict = {}
for currency in x_rates.columns[1:]:
    df_name = currency
#     col_names.append(df_name)
    df = x_rates[["date", currency]].copy()
    df = df[df[currency].notna()]
    df["log_rate"] = np.log(df.iloc[:,1]/df.iloc[:,1].shift()) # getting the log of the exchange rate # double check this is the correct way to get log
    df["rolling_mean_30"] = df[currency].rolling(30).mean()
    df["year"] = df["date"].dt.year
    df_dict[currency] = df

# currencies for comparison
high_freq = [df_dict["us_dollar"], df_dict["japanese_yen"], df_dict["uk_pound"], df_dict["australian_dollar"]]
low_freq = [df_dict["israeli_shekel"], df_dict["philippine_peso"], df_dict["malaysian_ringgit"], df_dict["romanian_leu"]]

# line graph showing Euro to each high frequency trading pairs
fig, ((ax1, ax2), (ax3, ax4)) = plt.subplots(nrows=2, ncols=2, figsize=(18,10))

for ax, currency in zip((ax1, ax2, ax3, ax4), high_freq):
    ax.plot(currency["date"], currency["rolling_mean_30"], linewidth=5, c=next(colors))
for ax in ax1, ax2, ax3, ax4:
    ax.set_xticks(["2000-01-04 00:00:00", "2005-01-04 00:00:00", "2010-01-04 00:00:00",
               "2015-01-04 00:00:00", "2020-01-04 00:00:00"])
    ax.set_xticklabels([2000,2005,2010,2015,2020], fontsize=18, weight="bold")
    ax.set(yticks=[])
#   ax.set_ylabel(str(currency)) # why not, i can get it to change to a string "currency", but not a variable
ax1.set_ylabel("US dollar", fontsize=18, weight="bold")
ax2.set_ylabel("Japanese yen", fontsize=18, weight="bold")
ax2.yaxis.set_label_position("right")
ax3.set_ylabel("UK pound", fontsize=18, weight="bold")
ax4.set_ylabel("Australian dollar", fontsize=18, weight="bold")
ax4.yaxis.set_label_position("right")
plt.subplots_adjust(wspace=0.01,hspace=0.01)
ax.text(0.08, 0.0, "© B McMinn" + " "*30 + "Source: Daria Chemkaeva",
        color = "#f0f0f0", backgroundcolor = "#4d4d4d", fontsize=14,
        transform=fig.transFigure)
fig.suptitle(" "*6 + "Euro to High Frequency Pairs" + " "*6,
             color = "#f0f0f0", backgroundcolor = "#4d4d4d", fontsize=35, weight="bold")
plt.show()

# line graph showing Euro to each low frequency trading pairs
fig, ((ax1, ax2), (ax3, ax4)) = plt.subplots(nrows=2, ncols=2, figsize=(18,10), sharex=True)
for ax, currency in zip((ax1, ax2, ax3, ax4), low_freq):
    ax.plot(currency["date"], currency["rolling_mean_30"], linewidth=5, c=next(colors))
for ax in ax1, ax2, ax3, ax4:
    ax.set_xticks(["2000-01-04 00:00:00", "2005-01-04 00:00:00", "2010-01-04 00:00:00",
               "2015-01-04 00:00:00", "2020-01-04 00:00:00"])
    ax.set_xticklabels([2000,2005,2010,2015,2020], fontsize=18, weight="bold")
    ax.set(yticks=[])
#   ax.set_ylabel(str(currency)) # why not, i can get it to change to a string "currency", but not a variable
ax1.set_ylabel("Israeli shekel", fontsize=18, weight="bold")
ax2.set_ylabel("Philippine peso", fontsize=18, weight="bold")
ax2.yaxis.set_label_position("right")
ax3.set_ylabel("Malaysian ringgit", fontsize=18, weight="bold")
ax4.set_ylabel("Romanian leu", fontsize=18, weight="bold")
ax4.yaxis.set_label_position("right")
# it would be best if I could match the ylabel color to the line color
plt.subplots_adjust(wspace=0.01,hspace=0.01)
ax.text(0.08, 0.0, "© B McMinn" + " "*30 + "Source: Daria Chemkaeva",
        color = "#f0f0f0", backgroundcolor = "#4d4d4d", fontsize=14,
        transform=fig.transFigure)
fig.suptitle(" "*6 + "Euro to Low Frequency Pairs" + " "*6,
             color = "#f0f0f0", backgroundcolor = "#4d4d4d", fontsize=35, weight="bold")
plt.show()


# ## **Euro & Traded Pairs Conclusion**
# 
# ### High Frequency Pairs
# * The US dollar, Japanese yen, and UK pound all show a tight ratio with the Euro in the early 2000s.
# * Both the Japanese yen and Australian dollar have a period in late 2011 where the exchange ratio gets tight and then rebounds. Is it really at the same time? Are there any other countries with the same pattern and what could the cause have been?
# ### Low Frequency Pairs
# * The Israeli shekel appears to have increased in value and the Romanian leu is definitely loosing value.
# * The shekel and leu appear to be fairly stable, or have low volatility.
# * The Israeli shekel and Malaysian ringgit also show the tight ratio of the early 2000s. This is seen to a lesser extent in the Phillipine peso.
# 
# ## **Alignments**
# Can I add a widget so that a person could pull the x value interactively?

# In[6]:


# %matplotlib widget
fig, ax = plt.subplots(figsize=(12,10))
ax1 = ax.twinx()
ax2 = ax.twinx()
ax3 = ax.twinx()
line = ax.plot(df_dict["us_dollar"]["date"], df_dict["us_dollar"]["rolling_mean_30"], 'tab:orange')
line_1 = ax1.plot(df_dict["us_dollar"]["date"], df_dict["japanese_yen"]["rolling_mean_30"], 'tab:blue')
line_2 = ax2.plot(df_dict["us_dollar"]["date"], df_dict["uk_pound"]["rolling_mean_30"], 'tab:green')
line_3 = ax3.plot(df_dict["us_dollar"]["date"], df_dict["australian_dollar"]["rolling_mean_30"], 'tab:red')
lines = [line, line_1, line_2, line_3]
for ax in ax, ax1, ax2, ax3:
    ax.set(yticks=[])
plt.show()


# ## **The Early Struggle and Other Alignments Conclusion**
# ### Stuff
# * Stuff
# * Stuff
# 
# ## **Volatility**
# Stuff

# In[7]:


# function to create df with year and annual volatility for every row
volatility_df_list = []
def volatizer(currency):
    annual_df_list = [currency[currency['year'] == y] for y in currency['year'].unique()] # list of annual dfs
    c_name = currency.columns[1]
    row_dict = {} # dictionary with year:annual_volatility as key:value 
    for frame in annual_df_list:
        year_name = frame.iat[0,4] # first cell of the "year" column, becomes the "year" key for row_dict
        annual_volatility = frame["log_rate"].std()*253**.5 # volatility measured by standard deviation * 253 trading days per year raised to the 0.5 power
        row_dict[year_name] = annual_volatility # dictionary with year:annual_volatility as key:value 
    df = pd.DataFrame.from_dict(row_dict, orient="index", columns=[c_name]) # new df from dictionary indexing on year
    volatility_df_list.append(df)

# apply volatizer to each currency df
for key in df_dict:
    df_dict[key] = volatizer(df_dict[key])
    
# create an annual volatility data frame with all 40 currencies
annual_volatility = pd.concat(volatility_df_list, axis=1)
annual_volatility["year"] = annual_volatility.index
# move year to position 0 and set index as int


# In[8]:


# bar charts comparing volatility for four low frequency pairs
# %matplotlib inline
low_freq = [annual_volatility["israeli_shekel"], annual_volatility["philippine_peso"], annual_volatility["malaysian_ringgit"], annual_volatility["romanian_leu"]]
years = annual_volatility.iloc[:,40]
fig, (ax1, ax2, ax3, ax4) = plt.subplots(nrows=1, ncols=4, figsize=(24,8), sharey=True)
colors = iter([plt.cm.Dark2(i) for i in range(4)])
for ax, currency in zip((ax1, ax2, ax3, ax4), low_freq):
    ax.barh(years, currency, height=0.65, color=next(colors), alpha=0.7)
    ax.set_xlim(0,0.21)
    ax.set_xticks([0.05, 0.1, 0.15, 0.2])
    ax.tick_params(axis="x", labelrotation=45)
#     ax.set_yticks([])
plt.subplots_adjust(wspace=0.01,hspace=0.01)
# ax.axvline(x=0.07, ymin=0.045, c="grey", alpha=0.5) # might use this later
ax.text(.25,.84,"Israeli shekel", color="green", fontsize=16, transform=fig.transFigure)
ax.text(.43,.84,"Philippine peso", color="orange", fontsize=16, transform=fig.transFigure)
ax.text(.62,.84,"Malaysian ringgit", color="purple", fontsize=16, transform=fig.transFigure)
ax.text(.83,.84,"Romanian leu", color="fuchsia", fontsize=16, transform=fig.transFigure)
ax.text(0.08, 0.0, "© B McMinn" + " "*310 + "Source: Daria Chemkaeva",
        color = "#f0f0f0", backgroundcolor = "#4d4d4d", fontsize=12,
        transform=fig.transFigure)
fig.suptitle(" "*43 + "Annual Volatility of Euro to Four Minor Currencies" + " "*43, color = "#f0f0f0", backgroundcolor = "#4d4d4d", fontsize=25, weight="bold")
plt.show()
# every other bar change alpha
# see if i can add y labels without the black ticks, maybe 4 horizontal lines that start and stop
# would be cool if i could match the label color to the graph color


# ## **Volatility Conclusion**
# * Stuff
# * Stuff
# 
# ## **Volatility Map**
# * Stuff

# In[9]:


# build and animate the map


# ## **Volatility Map Conclusion**
# * Stuff
# * Stuff
# 
# ## **Further Research**
# Stuff
# 
# ## **Conclusion**
# Stuff
# 
# ## **Citations**
# 1. Eichengreen, Barry (2019). Globalizing Capital: A History of the International Monetary System (3rd ed.). Princeton University Press. pp. 7, 79. ISBN 978-0-691-19390-8. JSTOR j.ctvd58rxg.
# 2. https://en.wikipedia.org/wiki/Bretton_Woods_system
# 3. Lowenstein, Roger (August 5, 2011). "The Nixon Shock". www.bloomberg.com. Bloomberg. Retrieved March 25, 2019.
# 4. Bank for International Settlements. "Foreign Exchange Turnover in April 2019." Accessed Oct. 3, 2020.
# 5. https://fxssi.com/the-most-traded-currency-pairs
# 6. https://www.kaggle.com/lsind18/euro-exchange-daily-rates-19992020 kaggle dataset

# In[10]:


# # need this to correctly get year ticks in DQ server
# years = mdates.YearLocator() # every year
# years_fmt = mdates.DateFormatter('%Y')
# for ax in ax1, ax2, ax3, ax4:
#     ax.xaxis.set_major_locator(years) # format the x ticks
#     ax.xaxis.set_major_formatter(years_fmt)
#     fmt_three_years = mdates.MonthLocator(interval = 36) # Major ticks every 3 years
#     ax.xaxis.set_major_locator(fmt_three_years)
#     ax.set(yticks=[],
#            xlabel="Year")


# In[11]:


# colors = iter([plt.cm.tab20(i) for i in range(20)])

# N = 50

# x = np.random.rand(N)
# y = np.random.rand(N)

# plt.scatter(x, y, c=[next(colors)])

# x = np.random.rand(N)
# y = np.random.rand(N)

# plt.scatter(x, y, c=[next(colors)])

# x = np.random.rand(N)
# y = np.random.rand(N)

# plt.scatter(x, y, c=[next(colors)])

# plt.show()