Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Enable ai-statistics plugin for AI routes by default #405

Merged
merged 1 commit into from
Jan 14, 2025

Conversation

CH3CHO
Copy link
Collaborator

@CH3CHO CH3CHO commented Jan 14, 2025

Ⅰ. Describe what this PR did

Enable ai-statistics plugin for AI routes by default.

image

image

apiVersion: extensions.higress.io/v1alpha1
kind: WasmPlugin
metadata:
  annotations:
    higress.io/wasm-plugin-description: Provides statistics of token usage, including
      logs, monitoring, and alerts.
    higress.io/wasm-plugin-icon: https://img.alicdn.com/imgextra/i1/O1CN018iKKih1iVx287RltL_!!6000000004419-2-tps-42-42.png
    higress.io/wasm-plugin-title: AI Statistics
  creationTimestamp: "2025-01-14T04:18:57Z"
  generation: 2
  labels:
    higress.io/resource-definer: higress
    higress.io/wasm-plugin-built-in: "true"
    higress.io/wasm-plugin-category: ai
    higress.io/wasm-plugin-name: ai-statistics
    higress.io/wasm-plugin-version: 1.0.0
  name: ai-statistics-1.0.0
  namespace: higress-system
  resourceVersion: "2945137"
  uid: bde30c13-b17d-4611-977a-ad52b0e228bd
spec:
  defaultConfigDisable: true
  failStrategy: FAIL_OPEN
  matchRules:
  - config:
      attributes:
      - apply_to_log: true
        key: question
        value: messages.@reverse.0.content
        value_source: request_body
      - apply_to_log: true
        key: answer
        rule: append
        value: choices.0.delta.content
        value_source: response_streaming_body
      - apply_to_log: true
        key: answer
        value: choices.0.message.content
        value_source: response_body
    configDisable: false
    ingress:
    - ai-route-qwen.internal
  - config:
      attributes:
      - apply_to_log: true
        key: question
        value: messages.@reverse.0.content
        value_source: request_body
      - apply_to_log: true
        key: answer
        rule: append
        value: choices.0.delta.content
        value_source: response_streaming_body
      - apply_to_log: true
        key: answer
        value: choices.0.message.content
        value_source: response_body
    configDisable: false
    ingress:
    - ai-route-qwen.fallback.internal
  phase: UNSPECIFIED_PHASE
  priority: 900
  url: oci://higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/ai-statistics:1.0.0

Ⅱ. Does this pull request fix one issue?

Ⅲ. Why don't you add test cases (unit test/integration test)?

Ⅳ. Describe how to verify it

Ⅴ. Special notes for reviews

@CH3CHO CH3CHO requested a review from johnlanni January 14, 2025 05:50
@CH3CHO CH3CHO requested a review from slievrly as a code owner January 14, 2025 05:50
Copy link
Contributor

@johnlanni johnlanni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@johnlanni johnlanni merged commit ac3b4fd into higress-group:main Jan 14, 2025
4 checks passed
@CH3CHO CH3CHO deleted the feat/ai-route-statistics branch January 14, 2025 05:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants