Add Mistral-7B-Instruct-v0.1 from huggingface. #2010

pranavsharma · 2023-10-25T00:38:58Z

Add Mistral-7B-Instruct-v0.1 from huggingface. See https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1

requirements.txt

pranavsharma · 2023-10-25T21:54:49Z

The A10G pipeline is failing even after disabling the test there. I can't even see the logs.

facebook-github-bot · 2023-10-26T13:15:14Z

@xuzhao9 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

pranavsharma · 2023-10-26T16:29:05Z

The A10G pipeline is failing even after disabling the test there. I can't even see the logs.

@xuzhao9 - is there anything that needs a fix here?

xuzhao9 · 2023-10-26T16:40:30Z

@pranavsharma The CPU test exceeds time limit (5 min), can you also help disable the CPU test?

torchbenchmark/models/mistral_7b_instruct/metadata.yaml

pranavsharma · 2023-10-27T22:14:04Z

@xuzhao9 - it's still failing with OOM.

msaroufim · 2023-10-28T00:26:22Z

torchbenchmark/util/framework/huggingface/model_factory.py

-    'phi_1_5' : (512, 512, 'AutoConfig.from_pretrained("microsoft/phi-1_5", trust_remote_code=True)', 'AutoModelForCausalLM')
+    'phi_1_5' : (512, 512, 'AutoConfig.from_pretrained("microsoft/phi-1_5", trust_remote_code=True)', 'AutoModelForCausalLM'),
+    # as per this page https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1 trust_remote_code=True is not required
+    'mistral_7b_instruct' : (512, 512, 'AutoConfig.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")', 'AutoModelForCausalLM')


reduce these numbers to avoid OOM, you're likely hitting the OOM because of how large the activations are

Reducing it to 128 doesn't work either.

pranavsharma · 2023-11-09T07:26:23Z

@msaroufim @xuzhao9 - how should we make progress on this? It's been pending for a while now.

xuzhao9 · 2023-11-10T00:46:46Z

Hi @pranavsharma , after 2 runs it still OOMs on A100 40GB. We need to either 1) slice/tune the model so that it will not OOM on A100 40GB, or 2) disable the A100 test, essentially not testing this model in our CI.

pranavsharma · 2023-11-15T19:54:11Z

Hi @pranavsharma , after 2 runs it still OOMs on A100 40GB. We need to either 1) slice/tune the model so that it will not OOM on A100 40GB, or 2) disable the A100 test, essentially not testing this model in our CI.

How do I disable A100 test?

xuzhao9 · 2023-11-15T20:41:03Z

torchbenchmark/models/mistral_7b_instruct/metadata.yaml

+train_benchmark: false
+train_deterministic: false
+not_implemented:
+  - device: NVIDIA A10G


Suggested change

- device: NVIDIA A10G

- device: NVIDIA A10G

- device: NVIDIA A100-SXM4-40GB

xuzhao9 · 2023-11-15T20:41:27Z

@pranavsharma Add the device name in the metadata.yaml as above.

pranavsharma · 2023-11-16T00:54:00Z

@xuzhao9 - does this look good?

xuzhao9 · 2023-11-16T05:09:15Z

torchbenchmark/util/framework/huggingface/model_factory.py

@@ -35,6 +35,8 @@
    'llama_v2_13b' : (512,512, 'AutoConfig.from_pretrained("meta-llama/Llama-2-13b-hf")', 'AutoModelForCausalLM'),
    'llama_v2_70b' : (512, 512, 'AutoConfig.from_pretrained("meta-llama/Llama-2-70b-hf")', 'AutoModelForMaskedLM'),
    'phi_1_5' : (512, 512, 'AutoConfig.from_pretrained("microsoft/phi-1_5", trust_remote_code=True)', 'AutoModelForCausalLM'),
+    # as per this page https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1 trust_remote_code=True is not required
+    'mistral_7b_instruct' : (128, 128, 'AutoConfig.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")', 'AutoModelForCausalLM')


A trailing comma is needed

Suggested change

'mistral_7b_instruct' : (128, 128, 'AutoConfig.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")', 'AutoModelForCausalLM')

'mistral_7b_instruct' : (128, 128, 'AutoConfig.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")', 'AutoModelForCausalLM'),

pranavsharma · 2023-11-23T00:56:18Z

Moved it to canary.

facebook-github-bot · 2023-12-18T22:02:40Z

@xuzhao9 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2023-12-19T04:07:52Z

@xuzhao9 merged this pull request in 97d6b17.

facebook-github-bot added the cla signed label Oct 25, 2023

pranavsharma had a problem deploying to docker-s3-upload October 25, 2023 00:39 — with GitHub Actions Error

Add Mistral-7B-Instruct-v0.1 from huggingface.

a933fde

pranavsharma force-pushed the mistral branch from fdadb37 to a933fde Compare October 25, 2023 01:06

pranavsharma had a problem deploying to docker-s3-upload October 25, 2023 01:07 — with GitHub Actions Failure

xuzhao9 reviewed Oct 25, 2023

View reviewed changes

requirements.txt Show resolved Hide resolved

xuzhao9 approved these changes Oct 25, 2023

View reviewed changes

Model is running out of memory on A10G. Hence excluding it.

75167dd

pranavsharma had a problem deploying to docker-s3-upload October 25, 2023 19:16 — with GitHub Actions Failure

pranavsharma had a problem deploying to docker-s3-upload October 25, 2023 19:17 — with GitHub Actions Error

pranavsharma had a problem deploying to docker-s3-upload October 26, 2023 13:14 — with GitHub Actions Failure

skip CPU test

b3bd7cc

pranavsharma had a problem deploying to docker-s3-upload October 26, 2023 16:58 — with GitHub Actions Failure

pranavsharma had a problem deploying to docker-s3-upload October 26, 2023 17:00 — with GitHub Actions Failure

xuzhao9 reviewed Oct 27, 2023

View reviewed changes

torchbenchmark/models/mistral_7b_instruct/metadata.yaml Outdated Show resolved Hide resolved

disable cpu

9b45b8c

pranavsharma had a problem deploying to docker-s3-upload October 27, 2023 17:09 — with GitHub Actions Failure

pranavsharma temporarily deployed to docker-s3-upload October 27, 2023 17:10 — with GitHub Actions Inactive

msaroufim reviewed Oct 28, 2023

View reviewed changes

pranavsharma had a problem deploying to docker-s3-upload November 1, 2023 23:04 — with GitHub Actions Failure

pranavsharma force-pushed the mistral branch from 914a4eb to 3bfd811 Compare November 1, 2023 23:54

pranavsharma had a problem deploying to docker-s3-upload November 1, 2023 23:56 — with GitHub Actions Failure

pranavsharma temporarily deployed to docker-s3-upload November 1, 2023 23:56 — with GitHub Actions Inactive

pranavsharma had a problem deploying to docker-s3-upload November 7, 2023 22:36 — with GitHub Actions Failure

xuzhao9 reviewed Nov 15, 2023

View reviewed changes

Fix devices, add smaller length and fix phi dep

a88756b

pranavsharma force-pushed the mistral branch from 3bfd811 to a88756b Compare November 16, 2023 00:48

pranavsharma had a problem deploying to docker-s3-upload November 16, 2023 00:53 — with GitHub Actions Failure

xuzhao9 reviewed Nov 16, 2023

View reviewed changes

Merge remote-tracking branch 'origin/main' into mistral

fd9b63f

pranavsharma force-pushed the mistral branch from 806656b to fd9b63f Compare November 16, 2023 07:20

pranavsharma had a problem deploying to docker-s3-upload November 16, 2023 07:21 — with GitHub Actions Failure

Add Mistral to canary

a973be2

pranavsharma had a problem deploying to docker-s3-upload November 23, 2023 00:55 — with GitHub Actions Failure

facebook-github-bot closed this in 97d6b17 Dec 19, 2023

facebook-github-bot added the Merged label Dec 19, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Mistral-7B-Instruct-v0.1 from huggingface. #2010

Add Mistral-7B-Instruct-v0.1 from huggingface. #2010

pranavsharma commented Oct 25, 2023

pranavsharma commented Oct 25, 2023

facebook-github-bot commented Oct 26, 2023

pranavsharma commented Oct 26, 2023

xuzhao9 commented Oct 26, 2023

pranavsharma commented Oct 27, 2023

msaroufim Oct 28, 2023 •

edited

Loading

pranavsharma Nov 2, 2023

pranavsharma commented Nov 9, 2023

xuzhao9 commented Nov 10, 2023

pranavsharma commented Nov 15, 2023

xuzhao9 Nov 15, 2023

xuzhao9 commented Nov 15, 2023 •

edited

Loading

pranavsharma commented Nov 16, 2023

xuzhao9 Nov 16, 2023

pranavsharma commented Nov 23, 2023

facebook-github-bot commented Dec 18, 2023

facebook-github-bot commented Dec 19, 2023

	- device: NVIDIA A10G
	- device: NVIDIA A10G
	- device: NVIDIA A100-SXM4-40GB

	'mistral_7b_instruct' : (128, 128, 'AutoConfig.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")', 'AutoModelForCausalLM')
	'mistral_7b_instruct' : (128, 128, 'AutoConfig.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")', 'AutoModelForCausalLM'),

Add Mistral-7B-Instruct-v0.1 from huggingface. #2010

Add Mistral-7B-Instruct-v0.1 from huggingface. #2010

Conversation

pranavsharma commented Oct 25, 2023

pranavsharma commented Oct 25, 2023

facebook-github-bot commented Oct 26, 2023

pranavsharma commented Oct 26, 2023

xuzhao9 commented Oct 26, 2023

pranavsharma commented Oct 27, 2023

msaroufim Oct 28, 2023 • edited Loading

Choose a reason for hiding this comment

pranavsharma Nov 2, 2023

Choose a reason for hiding this comment

pranavsharma commented Nov 9, 2023

xuzhao9 commented Nov 10, 2023

pranavsharma commented Nov 15, 2023

xuzhao9 Nov 15, 2023

Choose a reason for hiding this comment

xuzhao9 commented Nov 15, 2023 • edited Loading

pranavsharma commented Nov 16, 2023

xuzhao9 Nov 16, 2023

Choose a reason for hiding this comment

pranavsharma commented Nov 23, 2023

facebook-github-bot commented Dec 18, 2023

facebook-github-bot commented Dec 19, 2023

msaroufim Oct 28, 2023 •

edited

Loading

xuzhao9 commented Nov 15, 2023 •

edited

Loading