Skip to content

Commit

Permalink
📄 [jinja] Update README.md (#422)
Browse files Browse the repository at this point in the history
  • Loading branch information
xenova committed Dec 18, 2023
1 parent 7126cc0 commit 50679bc
Showing 1 changed file with 13 additions and 3 deletions.
16 changes: 13 additions & 3 deletions packages/jinja/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ const result = template.render({
// "<s>[INST] Hello, how are you? [/INST]I'm doing great. How can I help you today?</s> [INST] I'd like to show off how chat templating works! [/INST]"
```

### Transformers.js (coming soon)
### Transformers.js

First, install the `@huggingface/templates` and `@xenova/transformers` packages:

Expand All @@ -50,20 +50,30 @@ npm i @huggingface/templates
npm i @xenova/transformers
```

You can then render a list of chat messages using a tokenizer's `apply_chat_template` method.

```js
import { AutoTokenizer } from "@xenova/transformers";

// Load tokenizer from the Hugging Face Hub
const tokenizer = await AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1");

// Define chat messages
const chat = [
{ role: "user", content: "Hello, how are you?" },
{ role: "assistant", content: "I'm doing great. How can I help you today?" },
{ role: "user", content: "I'd like to show off how chat templating works!" },
];

const text = tokenizer.apply_chat_template(chat, { tokenize: false });
const text = await tokenizer.apply_chat_template(chat, { tokenize: false });
// "<s>[INST] Hello, how are you? [/INST]I'm doing great. How can I help you today?</s> [INST] I'd like to show off how chat templating works! [/INST]"
```

Notice how the entire chat is condensed into a single string. If you would instead like to return the tokenized version (i.e., a list of token IDs), you can use the following:

const input_ids = tokenizer.apply_chat_template(chat, { tokenize: true, return_tensor: false });
```js
const input_ids = await tokenizer.apply_chat_template(chat, { tokenize: true, return_tensor: false });
// [1, 733, 16289, 28793, 22557, 28725, 910, 460, 368, 28804, 733, 28748, 16289, 28793, 28737, 28742, 28719, 2548, 1598, 28723, 1602, 541, 315, 1316, 368, 3154, 28804, 2, 28705, 733, 16289, 28793, 315, 28742, 28715, 737, 298, 1347, 805, 910, 10706, 5752, 1077, 3791, 28808, 733, 28748, 16289, 28793]
```

For more information about chat templates, check out the transformers [documentation](https://huggingface.co/docs/transformers/main/en/chat_templating).

0 comments on commit 50679bc

Please sign in to comment.