1.5.1

This release contains the following:

• Support for Automatic evaluations (LLM as judge) using Inspect AI (https://github.com/UKGovernmentBEIS/inspect_ai)
• Advanced filters for filtering grouping traces
• Model based filters for metrics
• User ID based filters for metrics and traces
• Manual Evaluations is getting renamed as Annotations
• Bugfixes and performance improvements

Postgres Database Migration

Please run the following migration before updating & restarting your server:

RENAME COLUMN "expectedOutput" TO "annotatedOutput";

1.5.0

This release includes the following features:

• A revamp of the manual evaluations flow simplified for ease of use.
• Ability to import prompts from traces to the playground for iterating on traced prompts and completions against different models and model settings.
• Ability to save prompts from playground directly into the prompt registry.
• Bugfixes and query performance improvments

1.4.8

What's Changed

• Release 1.5.0 by @karthikscale3 in #136

Full Changelog: 1.4.7...1.4.8

1.4.7

What's Changed

• adding table exist checks by @dylanzuber-scale3 in #130

Full Changelog: 1.4.6...1.4.7

1.4.6

What's Changed

• Update README.md by @karthikscale3 in #124

Full Changelog: 1.4.5...1.4.6

1.4.5

• Save prompts from playground
• Image inputs
• Better visualization of logs

1.4.4

• Evaluation API bugfix
• Dark mode bugfix for time drop down filter

1.4.3

• Bugfixes

1.4.2

• Allow human evaluations tests for all traces
• Bugfixes

1.4.1

What's Changed

• Bump next from 14.1.0 to 14.1.1 by @dependabot in #93

Full Changelog: 1.4.0...1.4.1