From 610b856f85cae59492fd8b269e4b0bab46b2290c Mon Sep 17 00:00:00 2001 From: Rishin Raj Date: Fri, 27 Sep 2024 19:20:24 +0530 Subject: [PATCH] Updated FBS sample command (#135) Signed-off-by: Rishin Raj --- docs/source/quick_start.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/source/quick_start.md b/docs/source/quick_start.md index a081416d..1ece4836 100644 --- a/docs/source/quick_start.md +++ b/docs/source/quick_start.md @@ -86,8 +86,7 @@ When enabling continuous batching, batch size should not be specified. Users can leverage multi-Qranium and other supported features along with continuous batching. ```bash -python -m QEfficient.cloud.infer --model_name TinyLlama/TinyLlama_v1.1 --batch_size 3 --prompt_len 32 --ctx_len 128 --num_cores 16 --device_group [0] --prompt "My name is|The flat earth -theory is the belief that|The sun rises from" --mxfp6 --mos 1 --aic_enable_depth_first --full_batch_size 3 +python -m QEfficient.cloud.infer --model_name TinyLlama/TinyLlama_v1.1 --prompt_len 32 --ctx_len 128 --num_cores 16 --device_group [0] --prompt "My name is" --mxfp6 --mos 1 --aic_enable_depth_first --full_batch_size 4 ``` ## Python API