Is hierarchical configs desirable or boiler plate?

[ryotatomioka]

I am new to hydra and one of the things that confuses me is the duality between Python code and hydra config.

I have seen projects using hydra that have a configs folder with many subfolders that almost mirror the Python package structure. This pattern seems to be encouraged in the blog post: "Ideally, you want to compose your configuration just like composing code"
https://medium.com/pytorch/hydra-a-fresh-look-at-configuration-for-machine-learning-projects-50583186b710

However, this means that (1) default values are potentially specified in two places (in the code and config) and (2) forces developers to go back and forth between the code and configs to really understand what is going on.

So my question is: is this pattern considered desirable, or if it is done only because it is necessary to enable configuration of a deeply nested parameter.

As a concrete example, consider

# main.py
class DefaultCutoff:
    def __init__(self, threshold: float = 0.1):
        self.threshold = threshold

class DefaultPostProcessor:
    def __init__(self, cutoff):
        self.cutoff = cutoff

class App:
    def __init__(self, pre_processor = None, main_processor = None, post_processor = None):
        self.pre_processor = pre_processor or DefaultPreProcessor()
        self.main_processor = main_processor or DefaultMainProcessor()
        self.post_processor = post_processor or DefaultPostProcessor()

In order to be able to configure app.post_processor.cutoff.threshold I need to minimally have

app:
  _target_: main.App
  post_processor:
    _target_: main.DefaultPostProcessor
      cutoff:
        _target_: main.DefaultCutoff
        threshold: 0.1  # default value is duplicated!

In an ideal world, I would like to keep the default value in the code. It helps the reader understand the meaning of the value because it is close to the docstring. Also the reader doesn't have to jump to the corresponding config file to understand what a piece of code does most of the time.

However, this is not the case when we instantiate a nested object with _target_ field in the config file. _target_ field is currently required.

As an alternative, I can imagine putting everything in the default argument

# main.py
class DefaultCutoff:
    def __init__(self, threshold: float = 0.1):
        self.threshold = threshold

class DefaultPostProcessor:
    def __init__(self, cutoff_factory = DefaultCutoff):
        self.cutoff = cutoff_factory()

class App:
    def __init__(
        self,
        pre_processor_factory = DefaultPreProcessor,
        main_processor_factory = DefaultMainProcessor,
        post_processor_factory = DefaultPostProcessor,
    ):
        self.pre_processor = pre_processor_factory()
        self.main_processor = main_processor_factory()
        self.post_processor = post_processor_factory()

This is functionally the same as the first main.py but all default _target_ are specified in the code. Therefore, I believe hydra could allow a config file that looks like the following to put only the non-default value in the config

app:
  _target_: main.App
  post_processor_factory:
    _partial_: True
    cutoff_factory:
      _partial_: True
      threshold: 0.5

Here, I am using _partial_: True without _target_ to tell hydra to look up the default _target_ value in the code. Or even

app:
  _target_: main.App
  post_processor_factory:
    cutoff_factory:
      threshold: 0.5

because when the default value is a callable, _partial_: True is implied.

I believe something like this or similar (with a support from hydra) could make it easier for developers to understand the code (because of the presence of the default values), reduce duplication between code and configs, and reduce the config file to describe only the non-default values. I would love to hear your thoughs!

[odelalleau]

Are you familiar with structured configs? In many cases you should be able to leverage them to keep everything in the code. If this isn't enough, you may also be interested in hydra-zen.

[ryotatomioka]

Thank you for pointing me to the structured configs. I was more worried about the duplication of default values between the code and yaml file but we could write a decorator that automatically generates a structured config from a class. We can automatically insert the _target_ field. Is this something that you or anyone else have considered?

It would look like this:

from hydra.core.config_store import ConfigStore

@configurable
class Cutoff:
    def __init__(self, threshold: float):
        self.threshold = threshold

@configurable
class PostProcessor:
    def __init__(self, cutoff: Cutoff):
        self.cutoff = cutoff

@configurable
class PreProcessor:
    pass

@configurable
class MainProcessor:
    pass


@configurable
class App:
    def __init__(
        self,
        pre_processor: PreProcessor,
        main_processor: MainProcessor,
        post_processor: PostProcessor,
    ):
        self.pre_processor = pre_processor
        self.main_processor = main_processor
        self.post_processor = post_processor


cs = ConfigStore.instance()
cs.store(
    name="config",
    node={
        "my_app": App.Config(  # App.Config is auto-generated by the `@configurable` decorator
            pre_processor=PreProcessor.Config(),  # Similarly, PreProcessor.Config is auto-generated
            main_processor=MainProcessor.Config(),
            post_processor=PostProcessor.Config(
                cutoff=Cutoff.Config(threshold=0.1)  # default value
            ),
        ),
    },
)


@hydra.main(config_name="config")
def main(cfg: DictConfig) -> None:
    print(cfg)
    my_app = hydra.utils.instantiate(cfg.my_app)


if __name__ == "__main__":
    main()

Then we can run this script as python main.py or configure a nested parameter as python main.py my_app.post_processor.cutoff.threshold=0.2

[odelalleau]

I was more worried about the duplication of default values between the code and yaml file but we could write a decorator that automatically generates a structured config from a class. We can automatically insert the _target_ field. Is this something that you or anyone else have considered?

I believe hydra-zen has.

[ryotatomioka]

Thanks, now I see it.