Test and Improve Evaluation Flow

[alfredoihernandez]

Problem Statement

• Provide user (which doesn't know much about evaluations) with an easy to follow path to compare chatbots

Proposed Solution

• Run through and fix existing automation flow
• Setup generic flows which can be modifying easily depending on solution
• Evaluation flow can generate evaluation results both through AI Studio and locally
• PR will automatically trigger evaluation run and results get logged automatically somewhere
• Create/update README

References

• from @glav https://github.com/microsoft/content-summarization-accelerator/blob/main/getting-started-guide/docs/005-llmops/002-inner-loop/002-summarization-flows/002-evaluation-flow.md

Tasks

Beta Give feedback

No tasks being tracked yet.

Tasks

Beta Give feedback

No tasks being tracked yet.