From 9b94bc7b0f5b43fa96f92ab96fc37d133aa0bcd7 Mon Sep 17 00:00:00 2001 From: ayissi-msft Date: Mon, 7 Oct 2024 11:08:19 -0700 Subject: [PATCH] Add the README.md --- tools/python/model_validation/README.md | 33 +++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100644 tools/python/model_validation/README.md diff --git a/tools/python/model_validation/README.md b/tools/python/model_validation/README.md new file mode 100644 index 000000000..c449984a8 --- /dev/null +++ b/tools/python/model_validation/README.md @@ -0,0 +1,33 @@ +# ONNX Runtime GenAI Model Validation Example + +## Setup + +Clone this repository and navigate to the `tools/python/model_validation folder`. + +```bash +git clone https://github.com/microsoft/onnxruntime-genai.git +cd tools/python/model_validation +``` + +In the model_validation folder, you should find the validation_tool.py script, validation_config.json file, and this README.md. + +### Current Support +* Gemma +* Llama +* Mistral +* Phi +* Qwen + +### Usage - Build the Model +This step creates optimized and quantized ONNX models that run with ONNX Runtime GenAI. + +1. In the validation_config.json file, enter the supported Hugging Face model name. Models can be found here. +2. Include the path to the output folder, precision, and execution provider. + +Once the model is built, you can find it in path_to_output_folder/{model_name}. This should include the ONNX model data and tokenizer. + +### Run the Model Validation Script +```bash +python validation_tool.py -j validation_config.json +``` +