Skip to content

Commit

Permalink
Updated FBS sample command (quic#135)
Browse files Browse the repository at this point in the history
Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>
  • Loading branch information
quic-rishinr authored Sep 27, 2024
1 parent bbd96d3 commit 610b856
Showing 1 changed file with 1 addition and 2 deletions.
3 changes: 1 addition & 2 deletions docs/source/quick_start.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,7 @@ When enabling continuous batching, batch size should not be specified.
Users can leverage multi-Qranium and other supported features along with continuous batching.

```bash
python -m QEfficient.cloud.infer --model_name TinyLlama/TinyLlama_v1.1 --batch_size 3 --prompt_len 32 --ctx_len 128 --num_cores 16 --device_group [0] --prompt "My name is|The flat earth
theory is the belief that|The sun rises from" --mxfp6 --mos 1 --aic_enable_depth_first --full_batch_size 3
python -m QEfficient.cloud.infer --model_name TinyLlama/TinyLlama_v1.1 --prompt_len 32 --ctx_len 128 --num_cores 16 --device_group [0] --prompt "My name is" --mxfp6 --mos 1 --aic_enable_depth_first --full_batch_size 4
```
## Python API

Expand Down

0 comments on commit 610b856

Please sign in to comment.