Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question on how to Instantiate a list of objects with multiple configs #455

Open
ryokan0123 opened this issue Feb 19, 2024 · 2 comments
Open
Labels
enhancement New feature or request

Comments

@ryokan0123
Copy link

ryokan0123 commented Feb 19, 2024

Hi, I find jsonargparse incredibly useful for organizing code in ML experiments.
Thank you for creating such an excellent project.

I frequently encounter a use case where I need to pass a list of objects, each with separate config files.
This would allow me to experiment with combinations of complex objects.
However, I'm unsure if this is possible with the current version of jsonargparse.

For a minimal example, I'm looking to implement something like the following:

from dataclasses import dataclass
from typing import Any

import jsonargparse

@dataclass
class Foo:
    arg1: Any

@dataclass
class Bar:
    arg1: Any

@dataclass
class ComplexClass:
    foo: Foo
    bar: Bar


if __name__ == "__main__":
    parser = jsonargparse.ArgumentParser()
    parser.add_argument("--class_list", type=list[ComplexClass])

    args = parser.parse_args()
    args = parser.instantiate_classes(args)
    print(args)

And each ComplexClass would be configured through a separate config file like this:

foo:
  class_path: Foo
  init_args:
    arg: 1
bar:
  class_path: Bar
  init_args:
    arg: 2

Then the command would be:

python example.py --class_list+="complex_class.yaml"  --class_list+="complex_class2.yaml

Is there a way to achieve something like this?
I understand that configuring through a single config file is possible, possibly by using jsonnet to import multiple config files, but being able to directly specify multiple config files through command-line arguments would be convenient.

@mauvilsa
Copy link
Member

Currently that is not possible. Loading from subconfigs only works at argument level, not for each class in a list. A somewhat close alternative that would work now, although not ideal, is to use global config files with only the class you want to append to the list. The code would be:

from dataclasses import dataclass
from typing import Any

import jsonargparse

@dataclass
class Foo:
    arg1: Any

@dataclass
class Bar:
    arg1: Any

@dataclass
class ComplexClass:
    foo: Foo
    bar: Bar


if __name__ == "__main__":
    parser = jsonargparse.ArgumentParser()
    parser.add_argument("--config", action=jsonargparse.ActionConfigFile)
    parser.add_argument("--class_list", type=list[ComplexClass])

    args = parser.parse_args()
    args = parser.instantiate_classes(args)
    print(args)

be run as

python example.py --config=complex_class1.yaml --config=complex_class2.yaml

where each config would be like:

class_list+:
  - foo:
      arg1: 1
    bar:
      arg1: 2

Note that there are no class_path and init_args in the config. That is because dataclasses are not considered subclasses. This is until #287 is implemented and it becomes possible to select which class types should be considered or not subclasses.

@mauvilsa mauvilsa added the enhancement New feature or request label Feb 20, 2024
@ryokan0123
Copy link
Author

Loading from subconfigs is only effective at the argument level, not for individual classes within a list.

I see. This clarifies my question.

I will try the alternative solution you suggested. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants