build(deps): bump promptfoo from 0.90.1 to 0.92.1

[dependabot]

Bumps promptfoo from 0.90.1 to 0.92.1.

Release notes

Sourced from promptfoo's releases.

0.92.1

What's Changed

Features

• feat(providers): Add support for an optional responseSchema file to Google Gemini by @​aud

Fixes

• fix(provider): fal prompt config overrides by @​drochetti
• fix: count could be off if there was a test that wasn't recorded by @​sklein12
• fix: support relative paths by @​sklein12
• fix: Prompt ordering on tables by @​sklein12

Chores

• chore: delete empty file by @​sklein12
• chore: rename tables by @​sklein12
• chore(deps): bump openai from 4.67.1 to 4.67.2 by @​dependabot
• chore(deps-dev): bump @​aws-sdk/client-bedrock-runtime from 3.665.0 to 3.666.0 by @​dependabot

Documentation

• docs(contributing): expand guide for adding new providers by @​mldangelo

New Contributors

• @​aud made their first contribution in #1839

Full Changelog: promptfoo/promptfoo@0.92.0...0.92.1

0.92.0

What's Changed

Features

• Normalize eval results in db by @​sklein12 in promptfoo/promptfoo#1776 This introduces a new version of EvaluateSummary and json output

Site

• foundation model blog post by @​vsauter in promptfoo/promptfoo#1823
• site: custom blog index page by @​typpo in promptfoo/promptfoo#1824

Chores

• chore(build): allow-composite-ts by @​sklein12 in promptfoo/promptfoo#1825
• chore(cli): improve validation for extension hooks by @​mldangelo in promptfoo/promptfoo#1827

Full Changelog: promptfoo/promptfoo@0.91.3...0.92.0

0.91.3

What's Changed

• feat(redteam): add religion plugin by @​typpo in promptfoo/promptfoo#1822
• feat(provider-fal): allow prompt config overrides by @​drochetti in promptfoo/promptfoo#1815

... (truncated)

Commits

• 3a9dd47 chore: Bump version to 0.92.1
• e26bc4a fix: Prompt ordering on tables (#1843)
• b357875 fix: support relative paths (#1842)
• d1fd71e fix: count could be off if there was a test that wasn't recorded (#1841)
• eb41814 docs(providers): add Web Browser provider to index table
• b8ce2a6 feat(providers): Add support for an optional responseSchema file to google ...
• 797cd53 chore(deps-dev): bump @​aws-sdk/client-bedrock-runtime from 3.665.0 to 3.666.0...
• 3b6b56b chore(deps): bump openai from 4.67.1 to 4.67.2 (#1837)
• c751135 chore(providers): improve type safety in FalProvider callApi method
• d231660 fix(provider): fal prompt config overrides (#1835)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

[dependabot]

The following labels could not be found: npm, dependencies.

[yukinagae]

With promptfoo version 0.92.1, I go the below error and the CI failed:

$ npm run test

> genkitx-promptfoo@0.1.12 test
> tsx --test ./tests/*_test.ts

Prompt input is not a valid prompt schema: [
  {
    "code": "invalid_type",
    "expected": "string",
    "received": "undefined",
    "path": [
      "raw"
    ],
    "message": "Required"
  },
  {
    "code": "invalid_type",
    "expected": "string",
    "received": "undefined",
    "path": [
      "label"
    ],
    "message": "Required"
  }
]
Falling back to serialized JSON as raw prompt.
Prompt input is not a valid prompt schema: [
  {
    "code": "invalid_type",
    "expected": "string",
    "received": "undefined",
    "path": [
      "raw"
    ],
    "message": "Required"
  },
  {
    "code": "invalid_type",
    "expected": "string",
    "received": "undefined",
    "path": [
      "label"
    ],
    "message": "Required"
  }
]
Falling back to serialized JSON as raw prompt.
(node:31736) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
▶ promptfooScore
  ✖ should return a succeeding test result (56.395167ms)
    { code: 'SQLITE_ERROR' }

  ✖ should return a failing test result (1.67525ms)
    { code: 'SQLITE_ERROR' }

▶ promptfooScore (58.67575ms)
ℹ tests 2
ℹ suites 1
ℹ pass 0
ℹ fail 2
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 25471.09825

✖ failing tests:

test at tests/index_test.ts:2:1402
✖ should return a succeeding test result (56.395167ms)
  { code: 'SQLITE_ERROR' }

test at tests/index_test.ts:2:1402
✖ should return a failing test result (1.67525ms)
  { code: 'SQLITE_ERROR' }

[yukinagae]

• 0.91.3: ✅
• 0.92.0: 🆖
• 0.92.1: 🆖

[yukinagae]

changelog between 0.91.3 and 0.92.0

promptfoo/promptfoo@0.91.3...0.92.0

[yukinagae]

This big change might be the reason: promptfoo/promptfoo#1776

[yukinagae]

$ cd ~/.promptfoo/
$ ll
total 63520
drwxr-xr-x  552 yuki.nagae  staff     17664 Sep 26 14:11 cache
-rw-r--r--    1 yuki.nagae  staff         0 Sep 18 16:21 evalLastWritten
-rw-r--r--    1 yuki.nagae  staff  31608832 Oct  9 22:18 promptfoo.db
-rw-r--r--    1 yuki.nagae  staff        13 Sep  3 16:21 promptfoo.yaml
$ sqlite3 promptfoo.db
SQLite version 3.37.0 2021-12-09 01:34:53
Enter ".help" for usage hints.
sqlite> .tables
__drizzle_migrations  evals_to_datasets     prompts
datasets              evals_to_prompts      tags
eval_results          evals_to_providers
evals                 evals_to_tags

[dependabot]

OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting @dependabot ignore this major version or @dependabot ignore this minor version. You can also ignore all major, minor, or patch releases for a dependency by adding an ignore condition with the desired update_types to your config file.

If you change your mind, just re-open this PR and I'll resolve any conflicts on it.