Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
KuuCi authored Aug 31, 2023
1 parent 5f0b95d commit 970db51
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion examples/end-to-end-examples/support_chatbot/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

In this tutorial, we will be creating an application that answers questions about the MosaicML composer codebase. The basic structure of this application will be a retrieval question answering system where the user will provide the chatbot with a question, and then a language model will answer the question based on the retrieved text. See some [great](https://python.langchain.com/en/latest/modules/chains/index_examples/vector_db_qa.html#retrieval-question-answering) [materials](https://blog.langchain.dev/langchain-chat/) from [LangChain](https://python.langchain.com/en/latest/index.html) for more exploration on this type of application.

By default the model that is used throughout is [MPT-7b](https://huggingface.co/mosaicml/mpt-7b), a 7-billion parameter large language model trained by MosaicML. See [our blog](https://www.mosaicml.com/blog/mpt-7b) for more details. We have also released a more powerful, 30-billion parameter model, which you can swap in to this example very easily. See [MPT-30b](https://huggingface.co/mosaicml/mpt-30b) for more details. To swap in the 30b model, all you need to do is change all occurrences of `mpt-7b` to `mpt-30b`. The important ones to change are the `model.pretrained_model_name_or_path` and the `tokenizer.name`, in the various [finetune](./mcli-yamls/finetune/) yamls. The other occurrences are just in names of runs and save folders. Depending on your hardware, and particularly if you get a CUDA `c10` error, you may also need to change `device_train_microbatch_size` from `auto` to `1` in the [finetune](./mcli-yamls/finetune/) yamls.
By default the model that is used throughout is [MPT-30b](https://huggingface.co/mosaicml/mpt-30b), a 30-billion parameter large language model trained by MosaicML. See [our blog](https://www.mosaicml.com/blog/mpt-30b) for more details. Depending on your hardware, and particularly if you get a CUDA `c10` error, you may also need to change `device_train_microbatch_size` from `auto` to `1` in the [finetune](./mcli-yamls/finetune/) yamls.

![demo](web_app_screenshot.png)

Expand Down

0 comments on commit 970db51

Please sign in to comment.