#!/usr/bin/env python
# coding: utf-8

# <p align="center">
#     <img src="https://user-images.githubusercontent.com/46355364/220746807-669cdbc1-ac67-404c-b0bb-4a3d67d9931f.jpg" alt="Logo">
# </center>
# 
# [![GitHub Sponsors](https://img.shields.io/badge/Sponsor_this_Project-grey?logo=github)](https://github.com/sponsors/JerBouma)
# [![Documentation](https://img.shields.io/badge/Documentation-grey?logo=readme)](https://www.jeroenbouma.com/projects/financedatabase)
# [![Supported Python Versions](https://img.shields.io/pypi/pyversions/financedatabase)](https://pypi.org/project/financedatabase/)
# [![PYPI Version](https://img.shields.io/pypi/v/financedatabase)](https://pypi.org/project/financedatabase/)
# [![PYPI Downloads](https://static.pepy.tech/badge/financedatabase/month)](https://pepy.tech/project/financedatabase)
# 
# The **FinanceDatabase** serves the role of providing anyone with any type of financial product categorisation entirely for free. To be able to achieve this, the FinanceDatabase relies on involvement from the community to add, edit and remove tickers over time. This is made easy enough that anyone, even with a lack of coding experience can contribute because of the usage of CSV files that can be manually edited. I'd like to invite you to go to the **[Contributing Guidelines](https://github.com/JerBouma/FinanceDatabase/blob/main/CONTRIBUTING.md)** to understand how you can help. Thank you!
# 
# As a private investor, the sheer amount of information that can be found on the internet is rather daunting. Trying to 
# understand what type of companies or ETFs are available is incredibly challenging with there being millions of
# companies and derivatives available on the market. Sure, the most traded companies and ETFs can quickly be found
# simply because they are known to the public (for example, Microsoft, Tesla, S&P500 ETF or an All-World ETF). However, 
# what else is out there is often unknown.
# 
# **This database tries to solve that**. It features 300.000+ symbols containing Equities, ETFs, Funds, Indices, 
# Currencies, Cryptocurrencies and Money Markets. It therefore allows you to obtain a broad overview of sectors,
# industries, types of investments and much more.
# 
# The aim of this database is explicitly _not_ to provide up-to-date fundamentals or stock data as those can be obtained 
# with ease (with the help of this database) by using the [FinanceToolkit](https://github.com/JerBouma/FinanceToolkit). Instead, it gives insights into the products 
# that exist in each country, industry and sector and gives the most essential information about each product. With 
# this information, you can analyse specific areas of the financial world and/or find a product that is hard to find.
# 

# ## Table of Contents
# 
# 1. [Installation](#installation)
# 2. [Quick Start](#quickstart)
# 3. [Understanding the available options](#understanding-the-available-options)
# 4. [Collecting information from the database](#collecting-information-from-the-database)
#    1. [Equities](#equities)
#    2. [ETFs](#etfs)
#    3. [Funds](#funds)
#    4. [Indices](#indices)
#    5. [Currencies](#currencies)
#    6. [Cryptocurrencies](#cryptocurrencies)
#    7. [Money Markets](#moneymarkets)
# 5. [Searching the database in detail](#searching-the-database-in-detail)

# # Installation
# To install the FinanceDatabase it simply requires the following:
# 
# ```
# pip install financedatabase -U
# ```
# 
# Then within Python use:
# 
# ```python
# import financedatabase as fd
# ```

# In[1]:


import financedatabase as fd


# # Quickstart

# Same methods apply to all other asset classes as well. Columns may vary.

# In[2]:


# Initialize the Equities database
equities = fd.Equities()

# Obtain all countries from the database
equities_countries = equities.options("country")

# Obtain all sectors from the database
equities_sectors = equities.options("sector")

# Obtain all industry groups from the database
equities_industry_groups = equities.options("industry_group")

# Obtain all industries from a country from the database
equities_germany_industries = equities.options("industry", country="Germany")

# Obtain a selection from the database
equities_united_states = equities.select(country="United States")

# Obtain a detailed selection from the database
equities_usa_machinery = equities.select(
    country="United States", industry="Machinery"
)

# Search specific fields from the database
equities_uk_biotech = equities.search(
    country="United Kingdom", summary="biotech", exchange="LSE"
)

# Search specific fields from the database with lists
equities_media_services = equities.search(
    industry="Interactive Media & Services",
    country="United States",
    market_cap=["Large Cap", "Mega Cap"]
)

# Use the tickers to obtain data via the Finance Toolkit
telecomunication_services = equities.search(
    industry="Diversified Telecommunication Services",
    country="United States",
    market_cap="Mega Cap",
    exclude_exchanges=True)

toolkit = telecomunication_services.to_toolkit(
    api_key="FINANCIAL_MODELING_PREP_KEY",
    start_date="2000-01-01",
    progress_bar=False
)

# For example, obtain the historical data
historical_data = toolkit.get_historical_data()


# # Understanding the available options

# Understanding which countries, sectors, industries and categories exist is important to be able to search the database properly. Not only to understand the focus a specific the country but also to understand which area holds the most data. The output of all functionalities is cut off in this README for illustration purposes.
# 
# With `obtain_options` all possible options are given per column. This is useful as it doesn't require loading the larger data files. For example, obtaining all options for equities is done as follow:

# In[3]:


# Obtain all possible options for equities
fd.obtain_options("equities")


# Then, when initalising the equities dataset, the unique countries, sectors and industries of all equities in the database with any combination:

# In[4]:


# Initialize the Equities database
equities = fd.Equities()


# For countries, you will find the following list if you print `equities_countries`

# In[5]:


# Obtain all countries from the database
equities.options("country")


# For sectors, you will find the following list if you print `equities_sectors`:

# In[6]:


# Obtain all sectors from the database
equities.options("sector")


# For industry groups, you will find the following list if you print `equities_industry_groups`:

# In[7]:


equities.options("industry_group")


# For industries, you will find the following list if you print `equities_industries`:

# In[8]:


# Obtain all industries from the database
equities.options("industry")


# When you wish to get country, sector or industry specific lists, you can use the related `country`, `sector` and `industry` tags as also found in the help window with `help(equities.options)`

# In[9]:


help(equities.options)


# For example, if I wish to know all available industries within the sector "Basic Materials" in the country United States I can use

# In[10]:


# Obtain a filtered selection of available industries
equities.options(selection="industry", country="United States", sector="Materials")


# This also extends further if you are looking into a different category. For example, find all available currencies by using 

# In[11]:


# Initialize the Currencies database
currencies = fd.Currencies()

# Obtain all available currencies
currencies.options(selection="base_currency")


# But also when it comes to `etfs` with

# In[12]:


# Initialize the ETFs database
etfs = fd.ETFs()

# Obtain all availables categories
etfs.options(selection="category")


# # Collecting information from the database

# ## Equities

# If you wish to collect data from all equities you can use the following:

# In[13]:


equities = fd.Equities()

equities.select()


# This returns approximately 20.000 different equities. Note that by default, only the American exchanges are selected. These are symbols like `TSLA` (Tesla) and `MSFT` (Microsoft) that tend to be recognized by a majority of data providers and therefore is the default. To disable this, you can set the `exclude_exchanges` argument to `False` which then results in approximately 155.000 different symbols. Find a more elaborate explanation with `help(equities.select)`:

# In[14]:


help(equities.select)


# As an example, in [Understanding the available options](#understanding-the-available-options), we've used `equities.options(selection='industry', country="United States", sector="Materials")` which allowed us to look at a specific industry in the United States. So with this information in hand, I can now query the industry `Metals & Mining` as follows:
# 

# In[15]:


metals_and_mining = equities.search(industry="Metals & Mining", country="United States", market_cap="Large Cap", exclude_exchanges=True)

metals_and_mining


# The companies found from the Finance Database can be used to feed into the Finance Toolkit. For example the companies as found above can be used to obtain the income statements of all companies in the Metals & Mining industry in the United States by using the `to_toolkit` functionality.
# 
# Get an API key from Financial Modeling Prep **[here](https://site.financialmodelingprep.com/developer/docs/pricing/jeroen/)**. Note that the Free version only gets you 5 years of data and no quarterly statements but this link offers a 15% discount while also supporting the project.

# In[16]:


companies = metals_and_mining.to_toolkit(api_key="FINANCIAL_MODEL_PREP_KEY", start_date="2000-01-01", quarterly=False)

companies.get_profile()


# ## ETFs

# If you wish to collect data from all etfs you can use the following:

# In[17]:


etfs = fd.ETFs()

etfs.select()


# This returns approximately 2.500 different ETFs. Note that by default, only the American exchanges are selected. These are symbols like `SPY` (SPDR S&P 500 ETF Trust) and `VTI` (Vanguard Total Stock Market Index Fund ETF) that tend to be recognized by a majority of data providers and therefore is the default. To disable this, you can set the `exclude_exchanges` argument to `False` which then results in approximately 35.000 different symbols. Find a more elaborate explanation with `help(etfs.select)`:

# In[18]:


help(etfs.select)


# With this information in hand, and having seen the available options within [Understanding the available options](#understanding-the-available-options), we can specify the selection as follows:

# In[19]:


etfs.select(category="Developed Markets")


# ## Funds

# If you wish to collect data from all funds you can use the following:

# In[20]:


funds = fd.Funds()

funds.select()


# This returns approximately 30.000 different Funds. Note that by default, only the American exchanges are selected. These are symbols that tend to be recognized by a majority of data providers and therefore is the default. To disable this, you can set the `exclude_exchanges` argument to `False` which then results in approximately 55.000 different symbols. Find a more elaborate explanation with `help(funds.select)`:
# 

# In[21]:


help(funds.select)


# With this information in hand, and having seen the available options within [Understanding the available options](#understanding-the-available-options), we can specify the selection as follows:

# In[22]:


funds.options(selection="category_group")


# In[23]:


funds.options(selection="category", category_group="Equities")


# In[24]:


funds.select(category_group="Equities", category="China")


# ## Indices

# If you wish to collect data from all indices you can use the following:

# In[25]:


indices = fd.Indices()

indices.select()


# This returns approximately 60.000 different indices. Note that by default, only the American exchanges are selected. These are symbols like `^GSPC` (S&P 500) that tend to be recognized by a majority of data providers and therefore is the default. To disable this, you can set the `exclude_exchanges` argument to `False` which then results in approximately 90.000 different symbols. Find a more elaborate explanation with `help(indices.select)`:

# In[26]:


help(indices.select)


# ## Currencies

# If you wish to collect data from all currencies you can use the following:

# In[27]:


currencies = fd.Currencies()

currencies.select()


# This returns approximately 2.500 different currencies. Find a more elaborate explanation with `help(currencies.select)`:

# In[28]:


# Help Window
help(currencies.select)


# With this information in hand, and having seen the available options within [Understanding the available options](#understanding-the-available-options), we can specify the selection as follows:
# 

# In[29]:


currencies.select(base_currency="AED")


# ## Cryptocurrencies

# If you wish to collect data from all cryptocurrencies you can use the following:

# In[30]:


cryptos = fd.Cryptos()

cryptos.select()


# This returns approximately 3.000 different cryptocurrencies. Find a more elaborate explanation with `help(cryptos.select)`:

# In[31]:


help(cryptos.select)


# With this information in hand, and having seen the available options within [Understanding the available options](#understanding-the-available-options), we can specify the selection as follows. Which returns a total of 5 combination of cryptocurrencies that include the ETH.

# In[32]:


cryptos.select(crypto="ETH")


# ## Moneymarkets

# If you wish to collect data from all money markets you can use the following:

# In[33]:


moneymarkets = fd.Moneymarkets()

moneymarkets.select()


# This returns approximately 3.000 different money markets. Find a more elaborate explanation with `help(fd.select_moneymarkets)`:
# 

# In[34]:


help(moneymarkets.select)


# In[35]:


moneymarkets.select(category="Wells Fargo Funds Trust")


# # Searching the database in detail

# All asset classes have the capability to search each column with `search`, for example `equities.search()`. Through how this functionality is developed you can define multiple columns and search throughoutly. For example:

# In[36]:


# Collect all Equities
equities = fd.Equities()

# Search Multiple Columns
equities.search(summary='automotive', currency='USD', country='Germany')


# In[37]:


# Collect all Moneymarkets
moneymarkets = fd.Moneymarkets()

# Search Multiple Columns
moneymarkets.search(name="treasury fund", market="us_market")


# In[38]:


# Collect all Funds
funds = fd.Funds()

# Search Multiple Columns
funds.search(category_group="Fixed Income", category="Bonds", summary="United States")