Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mhof dev lr scheduler #898

Merged
merged 4 commits into from
Dec 10, 2024
+16 −3
Merged

Mhof dev lr scheduler #898

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
0103913
benchmark lr_scheduler
smilesun Dec 4, 2024
b0279dc
Merge branch 'lr_scheduler' into mhof_dev_lr_scheduler
smilesun Dec 4, 2024
7ac297a
Merge branch 'mhof_dev' into mhof_dev_lr_scheduler
smilesun Dec 10, 2024
c4d8bfc
Update aistat_trainer_combo_dial_irm_erm_mhof.yaml
smilesun Dec 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 16 additions & 3 deletions examples/benchmark/aistat_trainer_combo_dial_irm_erm_mhof.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,17 @@ domainlab_args:
zx_dim: 0
zy_dim: 64
zd_dim: 64
opt: AdamW
lr_scheduler: CosineAnnealingLR


Shared params:
opt:
distribution: categorical
datatype: str
values:
- AdamW
- Adam

nb4reg_over_task_ratio:
distribution: categorical # name of the distribution
datatype: int
Expand Down Expand Up @@ -79,14 +86,14 @@ fixed_multplier_irm_dial:
trainer: irm_dial
shared:
- gamma_reg
- nb4reg_over_task_ratio
- opt

feedforward_irm_dial:
model: erm
trainer: hyperscheduler_irm_dial
shared:
- gamma_reg
- nb4reg_over_task_ratio
- opt

mhof_irm_dial_erm:
model: erm
Expand All @@ -98,7 +105,13 @@ mhof_irm_dial_erm:
nb4reg_over_task_ratio: 0
str_setpoint_ada: "SliderAnyComponent()"
shared:
- opt
- k_i_gain_ratio
- mu_init
- mu_clip
- exp_shoulder_clip

erm:
model: erm
shared:
- opt
Loading