Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Swap default model from mistral to granite-7b-lab #311

Closed
Gregory-Pereira opened this issue Apr 22, 2024 · 6 comments
Closed

Swap default model from mistral to granite-7b-lab #311

Gregory-Pereira opened this issue Apr 22, 2024 · 6 comments
Assignees
Labels
documentation Improvements or additions to documentation enhancement Improve on an existing feature or experience models Anything related to models (upgrades, swap, etc)

Comments

@Gregory-Pereira
Copy link
Collaborator

Summit is coming, and with it will be information tying the instructlab models to our organization. We should be using our own tools and assets, so we should swap over to using our default model of instructlab/granite-7b-lab. This should be referenced in all our documentation as well as our actual MS / recipes.

Still have yet to hear thoughts about this from @rhatdan @sallyom @lmilbaum

@MichaelClifford
Copy link
Collaborator

I have run both instructlab/granite-7b-lab and instructlab/merlinte-7b-lab with the main branch build of llamacpp_python server using the `MODEL_CHAT_FORMAT=openchat" for both the chat application and summarizer (the only two recipes on AI Lab where we'd make this change.

I can confirm that both recipes work with both models.

@MichaelClifford
Copy link
Collaborator

MichaelClifford commented Apr 24, 2024

issue to get these models included in the AI Lab model catalog containers/podman-desktop-extension-ai-lab#979

@rhatdan
Copy link
Member

rhatdan commented Apr 24, 2024

Can you open a PR to change the default to use granite-7b-lab?

@Gregory-Pereira Gregory-Pereira added models Anything related to models (upgrades, swap, etc) good first issue Good for newcomers documentation Improvements or additions to documentation enhancement Improve on an existing feature or experience and removed good first issue Good for newcomers labels Apr 25, 2024
@Gregory-Pereira Gregory-Pereira added this to the Summit release (v1.1) milestone Apr 30, 2024
@Gregory-Pereira
Copy link
Collaborator Author

Closing as this is dealt with in #350

@devop-mmcgrath
Copy link

Can anyone confirm this also means Granite will show up in the list of models? I saw a demo earlier today and noticed it was missing from Models -> all list.

@MichaelClifford
Copy link
Collaborator

@devop-mmcgrath Yeah, we have granite in our model list, and its also propagated into the AI Lab UI.

https://github.com/containers/ai-lab-recipes/tree/main/models

Screenshot 2024-05-03 at 11 04 40 AM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement Improve on an existing feature or experience models Anything related to models (upgrades, swap, etc)
Projects
Development

No branches or pull requests

4 participants