Is it possible to split (and merge) config files (not for config groups)?

[hogru]

I would like to split my config 'yaml' file into a couple of files. Not to represent different options/config groups but to have data/file related parameters in one file, training/model/architecture related parameters in a 2nd file etc. Each of those files could have config groups (which I know how to do, at least the basics that I need).

I found OmegaConf.merge() but I do not know if/how I can reproduce that in Hydra. Is there a way and if yes, could you point me in the right direction, please? Hydra 1.2 is installed if that helps.

[Jasha10]

Hi @hogru,
Here is an example using the Hydra defaults list to merge several yaml files into one file:

# conf/dataset.yaml
data: cifar10

# conf/hyperparameters.yaml
hparam1: 123

# conf/config.yaml
defaults:
  - hyperparameters
  - dataset
  - _self_

foo: bar

# my_app.py
from omegaconf import DictConfig, OmegaConf
import hydra

@hydra.main(version_base=None, config_path="conf", config_name="config")
def my_app(cfg: DictConfig) -> None:
    print(OmegaConf.to_yaml(cfg))

if __name__ == "__main__":
    my_app()

$ tree
├── conf
│   ├── config.yaml
│   ├── dataset.yaml
│   └── hyperparameters.yaml
└── my_app.py

$ python my_app.py
hparam1: 123
data: cifar10
foo: bar

[hogru]

Hi @Jasha10,

Wow, so easy. I did overthink that so much. Thank you for saving me a lot of time, I would have visited several rabbit holes without your quick help. Since I already had running code / configuration, I just had to add a yaml file and a line in the config.yaml (and amend the struct config / dataclass).

Thank you!