In [1]:

%load_ext autoreload
%autoreload 2

In [2]:

import os
from hydra.experimental import initialize, initialize_config_module, initialize_config_dir, compose
from omegaconf import OmegaConf

Initializing Hydra¶

There are several ways to initialize. See the API docs for full details. All methods support both a function call style that changes the global state, and a context style that cleans up when theg scope exits.

initialize()¶

Initializes Hydra and add the config_path to the config search path. config_path is relative to the parent of the caller, in this case it is realtive to the directory containing this Notebook.

In [3]:

with initialize(config_path="cloud_app/conf"):
    cfg = compose(overrides=["+db=mysql"])
    print(cfg)

{'db': {'driver': 'mysql', 'user': '???', 'pass': '???'}}

initialize_config_module()¶

Initializes Hydra and add the config_module to the config search path.
The config module must be importable (an __init__.py must exist at its top level)

In [4]:

with initialize_config_module(config_module="cloud_app.conf"):
    cfg = compose(overrides=["+db=mysql"])
    print(cfg)

{'db': {'driver': 'mysql', 'user': '???', 'pass': '???'}}

initialize_config_dir()¶

Initializes Hydra and add the config_path to the config search path. The config_path must be an absolute path on the file system.

In [5]:

abs_config_dir=os.path.abspath("cloud_app/conf")
with initialize_config_dir(config_dir=abs_config_dir):
    cfg = compose(overrides=["+db=mysql"])
    print(cfg)

{'db': {'driver': 'mysql', 'user': '???', 'pass': '???'}}

Global initialization¶

Calling each of the initilizaiton methods outside of a context changes the global state.

In [6]:

initialize(config_path="cloud_app/conf")
compose(overrides=["+db=mysql"])

Out[6]:

{'db': {'driver': 'mysql', 'user': '???', 'pass': '???'}}

Config composition¶

Below are some more interesting demonstrations of config composition

Config Structure¶

The __init__.py file is needed is needed to help Python find the config files in some scenarios (It can be empty).

conf/
├── application
│   ├── bananas.yaml
│   └── donkey.yaml
├── cloud_provider
│   ├── aws.yaml
│   └── local.yaml
├── db
│   ├── mysql.yaml
│   └── sqlite.yaml
├── environment
│   ├── production.yaml
│   └── testing.yaml
├── config.yaml
└── __init__.py

In [7]:

# compose application specific config (in this case the applicaiton is "donkey")
cfg=compose(overrides= ["+db=mysql", "+environment=production", "+application=donkey"])
print(OmegaConf.to_yaml(cfg))

db:
  driver: mysql
  user: mysql
  pass: r4Zn*jQ9JB1Rz2kfz
donkey:
  name: kong
  rank: king

In [8]:

# compose from config.yaml, this composes a bunch of defaults in:
cfg=compose(config_name="config.yaml")
print(OmegaConf.to_yaml(cfg))

db:
  driver: sqlite
  user: test
  pass: test
  file: test.db
cloud:
  name: local
  host: localhost
  port: 9876

In [9]:

import os
os.environ["AWS_API_KEY"] = "__SOMETHING_FROM_AN_ENV_VARIABLE__"

# compose from config.yaml and override from the default testing
# environment to production and cloud provider to aws.
# Also override the ami id

cfg=compose(
    config_name="config.yaml", 
    overrides=[
        "cloud_provider=aws",
        "environment=production",
        "cloud.ami_id=MY_AMI_ID",
    ]
)
print(OmegaConf.to_yaml(cfg, resolve=True))

db:
  driver: sqlite
  user: mysql
  pass: r4Zn*jQ9JB1Rz2kfz
  file: test.db
cloud:
  name: aws
  api_key: __SOMETHING_FROM_AN_ENV_VARIABLE__
  ami_id: MY_AMI_ID