Added codellama

[MaanavD]

Adding codellama model to canary.

(doesn't run on 16GB GPU)

[xuzhao9]

I am good with adding it to canary.
Just asking, does it run on single 40GB A100?

[MaanavD]

@xuzhao9 it runs on A100 :)

$ python run.py codellama -d cuda
Warning: The model codellama cannot be found at core set.
/workspace/bowbao/onnxbench/transformers/src/transformers/utils/hub.py:124: FutureWarning: Using TRANSFORMERS_CACHE is deprecated and will be removed in v5 of Transformers. Use HF_HOME instead.
warnings.warn(
config.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 637/637 [00:00<00:00, 3.35MB/s]
Running eval method from codellama on cuda in eager mode with input batch size 1 and precision fp16.
GPU Time per batch: 45.310 milliseconds
CPU Wall Time per batch: 45.340 milliseconds
Time to first batch: 75980.8640 ms
GPU 0 Peak Memory: 18.8482 GB
CPU Peak Memory: 1.4941 GB

[xuzhao9]

@MaanavD Nice! In this case, I suggest we add this model to models/ instead of canary. We should disable cpu test because it is too slow and will time out. For A10G, it should not OOM because peak GPU memory is 18GB (A10G has 24GB), but we could also disable it if there is OOM.

[xuzhao9]

LGTM

[facebook-github-bot]

@xuzhao9 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[facebook-github-bot]

@xuzhao9 merged this pull request in 4386604.