New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Question about SD loss in cmnist code #1

Open

kiranchari opened this issue May 24, 2021 · 0 comments

kiranchari commented May 24, 2021 •

edited

Loading

Hi, I just came across this very very interesting project of yours, I was quite excited while reading the paper on arxiv.

While looking at the implementation of Spectral Decoupling for cmnist, I saw that flags.sd_coef is multipled with both the train loss as well as the SD loss term. I am referring to this line: https://github.com/mohammadpz/Gradient_Starvation/blob/main/Figure_4_and_table_1/cmnist.py#L155

Is this the intended usage or should it be something like loss = train_nll + (flags.sd_coef * sd)?

Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment