Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BFCL] Add Llama-3.2-1B-Instruct, Llama-3.2-3B-Instruct, Llama-3.1-8B-Instruct, Llama-3.1-70B-Instruct #657

Merged
merged 19 commits into from
Oct 5, 2024

Conversation

HuanzhiMao
Copy link
Collaborator

@HuanzhiMao HuanzhiMao commented Sep 24, 2024

This PR adds the following new models to the leaderboard:

  • meta-llama/Llama-3.2-1B-Instruct
  • meta-llama/Llama-3.2-1B-Instruct-FC
  • meta-llama/Llama-3.2-3B-Instruct
  • meta-llama/Llama-3.2-3B-Instruct-FC
  • meta-llama/Llama-3.1-8B-Instruct
  • meta-llama/Llama-3.1-8B-Instruct-FC
  • meta-llama/Llama-3.1-70B-Instruct
  • meta-llama/Llama-3.1-70B-Instruct-FC

@HuanzhiMao HuanzhiMao added the BFCL-New Model Add New Model to BFCL label Sep 24, 2024
@HuanzhiMao HuanzhiMao linked an issue Sep 24, 2024 that may be closed by this pull request
@HuanzhiMao HuanzhiMao changed the title [BFCL] Add Meta-Llama-3.1-8B-Instruct, Meta-Llama-3.1-70B-Instruct [BFCL] Add Meta-Llama-3.2-1B-Instruct, meta-llama/Meta-Llama-3.2-3B-Instruct, Meta-Llama-3.1-8B-Instruct, Meta-Llama-3.1-70B-Instruct Sep 26, 2024
@HuanzhiMao HuanzhiMao changed the title [BFCL] Add Meta-Llama-3.2-1B-Instruct, meta-llama/Meta-Llama-3.2-3B-Instruct, Meta-Llama-3.1-8B-Instruct, Meta-Llama-3.1-70B-Instruct [BFCL] Add Meta-Llama-3.2-1B-Instruct, Meta-Llama-3.2-3B-Instruct, Meta-Llama-3.1-8B-Instruct, Meta-Llama-3.1-70B-Instruct Sep 26, 2024
@HuanzhiMao
Copy link
Collaborator Author

Update:
This PR will add the Llama 3.2 models as well, since they share the same tool-calling template as the Llama 3.1 models.

@HuanzhiMao HuanzhiMao changed the title [BFCL] Add Meta-Llama-3.2-1B-Instruct, Meta-Llama-3.2-3B-Instruct, Meta-Llama-3.1-8B-Instruct, Meta-Llama-3.1-70B-Instruct [BFCL] Add Llama-3.2-1B-Instruct, Llama-3.2-3B-Instruct, Llama-3.1-8B-Instruct, Llama-3.1-70B-Instruct Sep 26, 2024
Copy link
Contributor

@Fanjia-Yan Fanjia-Yan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ShishirPatil ShishirPatil merged commit f09f7a2 into ShishirPatil:main Oct 5, 2024
@HuanzhiMao HuanzhiMao deleted the llama branch October 5, 2024 05:55
ShishirPatil pushed a commit that referenced this pull request Oct 21, 2024
This PR updates the leaderboard to reflect the change in score due to
the following PR merge:

1. #660 
2. #661
3. #683
4. #679
5. #708 
6. #709
7. #701
8. #657 
9. #658 
10. #640 
11. #653
12. #642 
13. #696 
14. #667

Close #662.

Note: Some models (like `firefunction`, `functionary`,
`microsoft/phi`)are not included in this leaderboard update because we
don't have all the entries generated. We will add them back once we get
the full result generated.
VishnuSuresh27 pushed a commit to VishnuSuresh27/gorilla that referenced this pull request Nov 11, 2024
…-Instruct, Llama-3.1-70B-Instruct (ShishirPatil#657)

This PR adds the following new models to the leaderboard:

  - `meta-llama/Llama-3.2-1B-Instruct`
  - `meta-llama/Llama-3.2-1B-Instruct-FC`
  - `meta-llama/Llama-3.2-3B-Instruct`
  - `meta-llama/Llama-3.2-3B-Instruct-FC`
  - `meta-llama/Llama-3.1-8B-Instruct`
  - `meta-llama/Llama-3.1-8B-Instruct-FC`
  - `meta-llama/Llama-3.1-70B-Instruct`
  - `meta-llama/Llama-3.1-70B-Instruct-FC`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
BFCL-New Model Add New Model to BFCL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BFCL] Llama31 results
3 participants