llama v2 7b #1775
llama v2 7b #1775
Conversation
… into msaroufim/authsd
|
Ok weird there's an OOM error here even though this is passing locally on my A10G |
| - device: cuda | ||
| test: train | ||
| - device: cuda | ||
| test: example |
There was a problem hiding this comment.
I am wondering why cuda example is disabled? Is it because of OOM?
The example test will check whether the output tensor is deterministic across runs, and it will be useful for the accuracy tests.
There was a problem hiding this comment.
Yep at num_heads = 16 this was the only test that OOM'd because of some cloning - lemme get the error message
There was a problem hiding this comment.
There was a problem hiding this comment.
If cloning is an issue, you can add DEEPCOPY = False to disable it, similar to
Can you please add DEEPCOPY = False and try again?
There was a problem hiding this comment.
Alright trying it now in CI
There was a problem hiding this comment.
Ok with deep copy off it still ooms at the full model size
|
Could you please paste the command and output of running the model locally? |
msaroufim
force-pushed
the
msaroufim/llamav2
branch
from
e0f091b
to
9fbceda
Compare
|
@msaroufim has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@msaroufim merged this pull request in 196b3b8. |
No description provided.