Configuring a ConfigSource plugin

[miterion]

I am trying to add a few config variables to my ConfigSource plugin but haven't found a good solution for this yet. Is there any way I can hookup into the general hydra config somehow?

[LrWm3]

I was unable to find a way to hook it up into the general hydra configuration.

The only way I could get it to work was to have two hydra configurations, one which loaded config for my config source, and then called the next config which used it. However, this was quite slow working against a remote config-source back-end.

One option is to provide query parameters as arguments in the search path of your config source plugin.

hydra:
  searchpath:
    - wandb://my-entity/model-registry/model?cache=true&cleanup-unused-artifacts=true

I attempted this and although it works it leaves a lot to be desired.

Another option is to fall-back to environment variables.

Overall its a bit of a chicken and egg situation because the ConfigSource has to tell Hydra what configuration files and groups are even available for resolution before it is able to load the configuration in the first place.

That said I think it would be nice if the search path logic was expanded to provide the ability to more easily configure how search path plugins work in a way that would at least be a bit easier to work with. searchpath may only be specified from the CLI and the entrypoint config already so I believe this could apply to something like would work:

hydra:
  searchpath:
    - awssecrets://secret
    - wandb://my-entity/model-registry/model
  searchpath-plugins:
    wandb:
      kwarg1: kwval1
      kwarg2: kwval2