Skip to content

Commit

Permalink
fix format
Browse files Browse the repository at this point in the history
Signed-off-by: alabulei1 <vivian.xiage@gmail.com>
  • Loading branch information
alabulei1 authored Nov 2, 2023
1 parent 4c5cd77 commit 112214d
Showing 1 changed file with 2 additions and 3 deletions.
5 changes: 2 additions & 3 deletions docs/develop/rust/wasinn/llm-inference.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,9 +96,8 @@ You can use environment variables to configure the model execution.

| Option |Default |Function |
| -------|-----------|----- |
| |
LLAMA_LOG| 0 |The backend will print diagnostic information when this value is set to 1|
|LLAMA_N_CTX |512| The context length is the max number of tokens in the entire conversation|
| LLAMA_LOG | 0 |The backend will print diagnostic information when this value is set to 1|
|LLAMA_N_CTX |512| The context length is the max number of tokens in the entire conversation|
|LLAMA_N_PREDICT |512|The number of tokens to generate in each response from the model|

For example, the following command specifies a context length of 4k tokens, which is standard for llama2, and the max number of tokens in each response to be 1k. It also tells WasmEdge to print out logs and statistics of the model at runtime.
Expand Down

0 comments on commit 112214d

Please sign in to comment.