diff --git a/packages/jinja/README.md b/packages/jinja/README.md index 7d1debba9..eef8840b6 100644 --- a/packages/jinja/README.md +++ b/packages/jinja/README.md @@ -41,7 +41,7 @@ const result = template.render({ // "[INST] Hello, how are you? [/INST]I'm doing great. How can I help you today? [INST] I'd like to show off how chat templating works! [/INST]" ``` -### Transformers.js (coming soon) +### Transformers.js First, install the `@huggingface/templates` and `@xenova/transformers` packages: @@ -50,20 +50,30 @@ npm i @huggingface/templates npm i @xenova/transformers ``` +You can then render a list of chat messages using a tokenizer's `apply_chat_template` method. + ```js import { AutoTokenizer } from "@xenova/transformers"; +// Load tokenizer from the Hugging Face Hub const tokenizer = await AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1"); +// Define chat messages const chat = [ { role: "user", content: "Hello, how are you?" }, { role: "assistant", content: "I'm doing great. How can I help you today?" }, { role: "user", content: "I'd like to show off how chat templating works!" }, ]; -const text = tokenizer.apply_chat_template(chat, { tokenize: false }); +const text = await tokenizer.apply_chat_template(chat, { tokenize: false }); // "[INST] Hello, how are you? [/INST]I'm doing great. How can I help you today? [INST] I'd like to show off how chat templating works! [/INST]" +``` + +Notice how the entire chat is condensed into a single string. If you would instead like to return the tokenized version (i.e., a list of token IDs), you can use the following: -const input_ids = tokenizer.apply_chat_template(chat, { tokenize: true, return_tensor: false }); +```js +const input_ids = await tokenizer.apply_chat_template(chat, { tokenize: true, return_tensor: false }); // [1, 733, 16289, 28793, 22557, 28725, 910, 460, 368, 28804, 733, 28748, 16289, 28793, 28737, 28742, 28719, 2548, 1598, 28723, 1602, 541, 315, 1316, 368, 3154, 28804, 2, 28705, 733, 16289, 28793, 315, 28742, 28715, 737, 298, 1347, 805, 910, 10706, 5752, 1077, 3791, 28808, 733, 28748, 16289, 28793] ``` + +For more information about chat templates, check out the transformers [documentation](https://huggingface.co/docs/transformers/main/en/chat_templating).