AI Toolkit docs migration #7827
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…e-docs into zhshang/aitoolkit
|
@ntrogh I just went through and addressed all your 1st pass comments. In addition I added 2 new pages for features we are releasing this week. Also did some minor updates of contents. Please help review. |
ntrogh
requested changes
Dec 16, 2024
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
Co-authored-by: Nick Trogh <1908215+ntrogh@users.noreply.github.com>
|
@ntrogh I've updated and addressed all round 2 comments. pls help review. |
ntrogh
requested changes
Jan 2, 2025
|
|
||
| Bulk run feature in AI Toolkit allows you to run many prompts in batch mode. Compared to playground you can only run each prompt manually in order. | ||
| The bulk run feature in AI Toolkit allows you to run multiple prompts in batch. When you use the playground, you can only run one prompt manually at a time, in the order they're listed. Bulk run takes a dataset as input, where each row in the dataset has a prompt as the minimal requirement. Typically teh dataset has multiple rows. Once imported, you can select any prompt to run or run all prompts on the selected model. The responses will be displayed in the same dataset view. Executed dataset can be exported. |
There was a problem hiding this comment.
Suggested change
| The bulk run feature in AI Toolkit allows you to run multiple prompts in batch. When you use the playground, you can only run one prompt manually at a time, in the order they're listed. Bulk run takes a dataset as input, where each row in the dataset has a prompt as the minimal requirement. Typically teh dataset has multiple rows. Once imported, you can select any prompt to run or run all prompts on the selected model. The responses will be displayed in the same dataset view. Executed dataset can be exported. | |
| The bulk run feature in AI Toolkit allows you to run multiple prompts in batch. When you use the playground, you can only run one prompt manually at a time, in the order they're listed. Bulk run takes a dataset as input, where each row in the dataset has a prompt as the minimal requirement. Typically, the dataset has multiple rows. Once imported, you can select any prompt to run or run all prompts on the selected model. The responses will be displayed in the same dataset view. The results from running the dataset can be exported. |
|
|
||
| Navigate to `Bulk Run` in the `TOOLS` section in the tree view of AI Toolkit. You can select either a sample dataset to get familiar about the feature, or import a local JSONL file that has at least a field `query` to use as prompts. | ||
| 1. Open Bulk Run view |
There was a problem hiding this comment.
Suggested change
| 1. Open Bulk Run view | |
| 1. In the AI Toolkit view, select **TOOLS** > **Bulk Run** to open the Bulk Run view. |
|
|
||
| In the AI Toolkit view, select **TOOLS** > **Bulk Run** to open the Bulk Run view. |
There was a problem hiding this comment.
Suggested change
| In the AI Toolkit view, select **TOOLS** > **Bulk Run** to open the Bulk Run view. |
|
|
||
| ## Run any prompt | ||
| 1. Select dataset |
There was a problem hiding this comment.
Suggested change
| 1. Select dataset | |
| 1. Select either a sample dataset or import a local JSONL file that has a `query` field to use as prompts. |
|
|
||
| Once dataset is loaded in the bulk run view, you can find the table format of the imported dataset, with column of query and column of response. | ||
| You can select either a sample dataset to get familiar about the feature, or import a local JSONL file that has at least a field `query` to use as prompts. |
There was a problem hiding this comment.
Suggested change
| You can select either a sample dataset to get familiar about the feature, or import a local JSONL file that has at least a field `query` to use as prompts. |
|
|
||
| ## Monitor the evaluation job | ||
|
|
||
| Once an evaluation job is started, you can find its status from the evaluation job overview. Each evaluation job has link to the dataset used, logs from evaluation process, timestamp and details of the evaluation. | ||
| Once an evaluation job is started, you can find its status from the evaluation job view. | ||
|  |
There was a problem hiding this comment.
Suggested change
|  | |
|  | |
| Once an evaluation job is started, you can find its status from the evaluation job overview. Each evaluation job has link to the dataset used, logs from evaluation process, timestamp and details of the evaluation. | ||
| Once an evaluation job is started, you can find its status from the evaluation job view. | ||
|  | ||
| Each evaluation job has link to the dataset used, logs from evaluation process, timestamp and link to the details of the evaluation. |
There was a problem hiding this comment.
Suggested change
| Each evaluation job has link to the dataset used, logs from evaluation process, timestamp and link to the details of the evaluation. | |
| Each evaluation job has a link to the dataset that was used, logs from the evaluation process, timestamp, and a link to the details of the evaluation. |
| You can also select **Open In Data Wrangler** to open the data in `Data Wrangler` extension. |
There was a problem hiding this comment.
Suggested change
| You can also select **Open In Data Wrangler** to open the data in `Data Wrangler` extension. | |
| You can also select **Open In Data Wrangler** to open the data with the Data Wrangler extension. |
|
|
||
| You can filter the models in the model catalog by using the following criteria: | ||
| - Hosted by: | ||
| 1. use the filters to reuce the list of available models |
There was a problem hiding this comment.
Suggested change
| 1. use the filters to reuce the list of available models | |
| 1. Use the filters to reduce the list of available models |
| You can filter the models in the model catalog by using the following criteria: | ||
| - Hosted by: | ||
| 1. use the filters to reuce the list of available models | ||
|  |
There was a problem hiding this comment.
Put the screenshot after the list of filters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.