KenyonY · KenyonY · Jan 30, 2024 · Jan 29, 2024 · Jan 29, 2024 · Jan 30, 2024
diff --git a/.env b/.env
@@ -1,3 +1,5 @@
+# 建议使用WebUI进行配置，配置后可导出.env文件
+
 # Sample and explanation can be found in .env.example
 
 LOG_GENERAL=true
@@ -18,20 +20,21 @@ DEFAULT_REQUEST_CACHING_VALUE=false
 
 FORWARD_CONFIG=[{"base_url":"https://api.openai.com","route":"/","type":"openai"}]
 
-#OPENAI_API_KEY={"sk-xxx": 0}
-#FORWARD_KEY={"fk-xxxxxx": 0}
+#OPENAI_API_KEY=
+#FORWARD_KEY={"fk-0": 0, "fk-1": 1, "fk-2": 2}
+#LEVEL_MODELS={"1": ["gpt-4"], "2": ["gpt-3.5-turbo"]}
 
 
 # `REQ_RATE_LIMIT`: i.e., Request rate limit for specified routes, user specific
 # format: {route: ratelimit-string}
 # ratelimit-string format [count] [per|/] [n (optional)] [second|minute|hour|day|month|year] :ref:`ratelimit-string`: https://limits.readthedocs.io/en/stable/quickstart.html#rate-limit-string-notation
-REQ_RATE_LIMIT={"/v1/chat/completions":"100/2minutes", "/v1/completions":"60/minute;600/hour"}
+REQ_RATE_LIMIT={"/v1/chat/completions":"100/2minutes","/v1/completions":"60/minute;600/hour"}
 
 # Backend for rate limiting: [memory, redis, memcached, ...] :ref: https://limits.readthedocs.io/en/stable/storage.html#
 #REQ_RATE_LIMIT_BACKEND=redis://localhost:6379
 
 # `GLOBAL_RATE_LIMIT`: Limits all routes not specified by `REQ_RATE_LIMIT`. If not set, there's no limit by default.
-GLOBAL_RATE_LIMIT=100/minute
+GLOBAL_RATE_LIMIT=200/minute
 
 #`RATE_LIMIT_STRATEGY` Options: (fixed-window, fixed-window-elastic-expiry, moving-window) :ref: https://limits.readthedocs.io/en/latest/strategies.html
 # `fixed-window`: most memory efficient strategy; `moving-window`:most effective for preventing bursts but has higher memory cost.

diff --git a/.env.example b/.env.example
@@ -1,3 +1,5 @@
+# 建议使用WebUI进行配置，配置后可导出.env文件
+
 GENERAL_LOG=true
 OPENAI_LOG=true
 
@@ -18,6 +20,7 @@ FORWARD_CONFIG=[{"base_url":"https://api.openai.com","route":"/","type":"openai"
 
 #OPENAI_API_KEY={"sk-***":0,"***":0}
 #FORWARD_KEY={"fk-***":0,"***":0}
+#LEVEL_MODELS={"1": ["gpt-4"], "2": ["gpt-3.5-turbo"]}
 
 
 # `REQ_RATE_LIMIT`: 指定路由的请求速率限制（区分用户）

diff --git a/.github/workflows/docker-publish.yml b/.github/workflows/docker-publish.yml
@@ -60,7 +60,7 @@ jobs:
           file: ./webui.Dockerfile
           platforms: linux/amd64,linux/arm64
           push: true
-          tags: webui-${{ steps.meta.outputs.tags }}
+          tags: webui-${{ github.ref_name }}
           labels: ${{ steps.meta.outputs.labels }}
           cache-from: type=gha
           cache-to: type=gha,mode=max
diff --git a/Dockerfile b/Dockerfile
@@ -1,21 +1,26 @@
-FROM python:3.10-alpine
-LABEL maintainer="kunyuan"
+FROM python:3.10-slim
+LABEL maintainer="K.Y"
 ENV LC_ALL=C.UTF-8
 ENV LANG=C.UTF-8
 ENV TZ=Asia/Shanghai
-RUN apk update && \
-    apk add tzdata --no-cache && \
-    cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && \
-    apk del tzdata && \
-    mkdir -p /usr/share/zoneinfo/Asia/ && \
-    ln -s /etc/localtime /usr/share/zoneinfo/Asia/Shanghai
+
+RUN apt update && \
+    apt install -y tzdata && \
+    ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && \
+    dpkg-reconfigure -f noninteractive tzdata && \
+    apt clean && \
+    rm -rf /var/lib/apt/lists/*
 
 COPY . /home/openai-forward
 WORKDIR /home/openai-forward
-RUN apk add patch g++ libstdc++ leveldb-dev linux-headers --no-cache && \
-    pip install -e . --no-cache-dir && \
-    pip install "lmdb>=1.4.1" "plyvel>=1.5.0" --no-cache-dir && \
-    apk del g++ gcc && rm -rf /var/cache/apk/*
+RUN apt-get update && \
+    apt-get install -y patch g++ gcc libstdc++6 libtcmalloc-minimal4 libleveldb-dev cmake make build-essential && \
+    pip3 install -e . --no-cache-dir && \
+    apt-get remove -y patch g++ gcc cmake make build-essential && \
+    apt-get autoremove -y && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
+
 
 EXPOSE 8000
-ENTRYPOINT ["python", "-m", "openai_forward.__main__", "run"]
+ENTRYPOINT ["python3", "-m", "openai_forward.__main__", "run"]
diff --git a/README.md b/README.md
@@ -64,7 +64,6 @@
 
 ## 主要特性
 
-OpenAI-Forward 提供以下核心功能：
 
 - **全能转发**：可转发几乎所有类型的请求
 - **性能优先**：出色的异步性能
@@ -73,6 +72,7 @@ OpenAI-Forward 提供以下核心功能：
 - **实时响应日志**：提升LLMs可观察性
 - **自定义秘钥**：替代原始API密钥
 - **多目标路由**：转发多个服务地址至同一服务下的不同路由
+- **黑白名单**：可对指定IP进行黑白名单限制
 - **自动重试**：确保服务的稳定性，请求失败时将自动重试
 - **快速部署**：支持通过pip和docker在本地或云端进行快速部署
 
@@ -220,56 +220,25 @@ curl --location 'https://api.openai-forward.com/v1/images/generations' \
 (更多)
 
 
-
 <a>
    <img src="https://raw.githubusercontent.com/KenyonY/openai-forward/main/.github/images/separators/aqua.png" height=8px width="100%">
 </a>
 
 ## 配置
 
-### 命令行参数
-
-执行 `aifd run --help` 获取参数详情
 
-<details open>
-  <summary>Click for more details</summary>
-
-| 配置项       | 说明    | 默认值  |
-|-----------|-------|:----:|
-| --port    | 服务端口号 | 8000 |
-| --workers | 工作进程数 |  1   |
-
-</details>
+执行 `aifd run --webui` 进入配置页面 (默认服务地址 http://localhost:8001)
 
-### 环境变量详情
 
 你可以在项目的运行目录下创建 .env 文件来定制各项配置。参考配置可见根目录下的
 [.env.example](.env.example)文件
 
-| 环境变量                          | 说明                                                                   |                                 默认值                                 |
-|-------------------------------|----------------------------------------------------------------------|:-------------------------------------------------------------------:|
-| FORWARD_CONFIG                | 配置转发base url与 转发路由                                                   | [{"base_url":"https://api.openai.com","route":"/","type":"openai"}] |
-| OPENAI_API_KEY                | 配置OpenAI 接口风格的API密钥，支持使用多个密钥，通过逗号分隔                                  |                                  无                                  |
-| FORWARD_KEY                   | 设定用于代理的自定义密钥，多个密钥可用逗号分隔。如果未设置(不建议)，将直接使用 `OPENAI_API_KEY`            |                                  无                                  |
-| EXTRA_BASE_URL                | 用于配置额外代理服务的基础URL                                                     |                                  无                                  |
-| EXTRA_ROUTE_PREFIX            | 定义额外代理服务的路由前缀                                                        |                                  无                                  |
-| REQ_RATE_LIMIT                | 设置特定路由的用户请求速率限制 (区分用户)                                               |                                  无                                  |
-| GLOBAL_RATE_LIMIT             | 配置全局请求速率限制，适用于未在 `REQ_RATE_LIMIT` 中指定的路由                             |                                  无                                  |
-| RATE_LIMIT_STRATEGY           | 选择速率限制策略，选项包括：fixed-window、fixed-window-elastic-expiry、moving-window |                                  无                                  |
-| TOKEN_RATE_LIMIT              | 限制流式响应中每个token（或SSE chunk）的输出速率                                      |                                  无                                  |
-| PROXY                         | 设置HTTP代理地址                                                           |                                  无                                  |
-| LOG_CHAT                      | 开关聊天内容的日志记录，用于调试和监控                                                  |                               `false`                               |
-| CACHE_BACKEND                 | cache后端，支持内存后端和数据库后端，默认为内存后端，可选lmdb、leveldb数据库后端                     |                               `lmdb`                                |
-| CACHE_CHAT_COMPLETION         | 是否缓存/v1/chat/completions 结果                                          |                               `false`                               |
-| DEFAULT_REQUEST_CACHING_VALUE | 是否默认开启缓存                                                             |                               `false`                               |
-
-详细配置说明可参见 [.env.example](.env.example) 文件。(待完善)
-
-> 注意：如果你设置了 OPENAI_API_KEY 但未设置 FORWARD_KEY，客户端在调用时将不需要提供密钥。由于这可能存在安全风险，除非有明确需求，否则不推荐将
-> FORWARD_KEY 置空。
 
 ### 智能缓存
 
+开启缓存后，将会对指定路由的内容进行缓存，其中转发类型分别为`openai`与`general`两者行为略有不同，
+使用`general`转发时，默认会将相同的请求一律使用缓存返回，  
+使用`openai`转发时，在开启缓存后，可以通过OpenAI 的`extra_body`参数来控制缓存的行为，如
 
 **Python**
 

diff --git a/README_EN.md b/README_EN.md
@@ -77,6 +77,7 @@ OpenAI-Forward offers the following capabilities:
 - **User Traffic Control**: Customize request and Token rates.
 - **Real-time Response Logs**: Enhances observability of the call chain.
 - **Custom Secret Keys**: Replaces the original API keys.
+- **Black and White List**: IP-based black and white list restrictions can be implemented.
 - **Multi-target Routing**: Forwards to multiple service addresses under a single service to different routes.
 - **Automatic Retries**: Ensures service stability; will automatically retry on failed requests.
 - **Quick Deployment**: Supports fast deployment locally or on the cloud via pip and docker.
@@ -233,46 +234,16 @@ curl --location 'https://api.openai-forward.com/v1/images/generations' \
 
 ## Configuration
 
-### Command Line Arguments
-
-Execute `aifd run --help` to get details on arguments.
-
-<details open>
-  <summary>Click for more details</summary>
-
-| Configuration | Description | Default Value |
-|---------------|-------------|:-------------:|
-| --port       | Service port | 8000         |
-| --workers    | Number of working processes | 1 |
-
-</details>
-
-### Environment Variable Details
-
-You can create a .env file in the project's run directory to customize configurations. For a reference configuration, see the [.env.example](.env.example) file in the root directory.
-
-| Environment Variable  | Description                                                                                      |     Default Value      |
-|-----------------------|-------------------------------------------------------------------------------------------------|:----------------------:|
-| FORWARD_CONFIG                |                                                    | [{"base_url":"https://api.openai.com","route":"/","type":"openai"}] |
-| OPENAI_API_KEY        | Configure API key in OpenAI style, supports using multiple keys separated by commas              |          None          |
-| FORWARD_KEY           | Set a custom key for proxying, multiple keys can be separated by commas. If not set (not recommended), it will directly use `OPENAI_API_KEY` |          None          |
-| EXTRA_BASE_URL        | Configure the base URL for additional proxy services                                             |          None          |
-| EXTRA_ROUTE_PREFIX    | Define the route prefix for additional proxy services                                           |          None          |
-| REQ_RATE_LIMIT        | Set the user request rate limit for specific routes (user distinguished)                         |          None          |
-| GLOBAL_RATE_LIMIT     | Configure a global request rate limit applicable to routes not specified in `REQ_RATE_LIMIT`    |          None          |
-| RATE_LIMIT_STRATEGY   | Choose a rate limit strategy, options include: fixed-window, fixed-window-elastic-expiry, moving-window |          None          |
-| TOKEN_RATE_LIMIT      | Limit the output rate of each token (or SSE chunk) in a streaming response                      |          None          |
-| PROXY                 | Set HTTP proxy address                                                                           |          None          |
-| LOG_CHAT              | Toggle chat content logging for debugging and monitoring                                        |        `false`         |
-| CACHE_BACKEND         | Cache backend, supports memory backend and database backend. By default, it's memory backend, optional database backends are lmdb, and leveldb |         `lmdb`         |
-| CACHE_CHAT_COMPLETION | Whether to cache /v1/chat/completions results                                                    |        `false`         |
+Execute `aifd run --webui` to enter the configuration page (default service address http://localhost:8001).
 
 Detailed configuration descriptions can be seen in the [.env.example](.env.example) file. (To be completed)
 
-> Note: If you set OPENAI_API_KEY but did not set FORWARD_KEY, clients will not need to provide a key when calling. As this may pose a security risk, it's not recommended to leave FORWARD_KEY unset unless there's a specific need.
 
 ### Caching
 
+After enabling caching, the content of specified routes will be cached. The forwarding types are divided into `openai` and `general`, with slightly different behaviors for each.
+When using `general` forwarding, by default, the same requests will all be responded to using the cache.
+When using `openai` forwarding, after enabling caching, the caching behavior can be controlled through OpenAI's `extra_body` parameter, such as
 
 **Python**
 ```diff
@@ -302,7 +273,7 @@ curl https://smart.openai.com/v1/chat/completions \
 
 ```
 
-### Custom Keys
+### Custom Api Keys
 
 <details open>
   <summary>Click for more details</summary>

diff --git a/deploy.md b/deploy.md
@@ -90,7 +90,6 @@ docker run --env-file .env -d -p 8000:8000 beidongjiedeguang/openai-forward:late
 将映射宿主机的8000端口，通过`http://{ip}:8000`访问服务。  
 容器内日志路径为`/home/openai-forward/Log/`, 可在启动时将其映射出来。  
 
-注：同样可以在启动命令中通过-e传入环境变量OPENAI_API_KEY=sk-xxx作为默认api key  
 启用SSL同上.
 环境变量配置见[环境变量配置](README.md#配置)
 

diff --git a/deploy_en.md b/deploy_en.md
@@ -85,8 +85,7 @@ docker run --env-file .env -d -p 8000:8000 beidongjiedeguang/openai-forward:late
 This will map the host's 8000 port. Access the service via `http://{ip}:8000`.  
 The log path inside the container is `/home/openai-forward/Log/`. It can be mapped when starting up.
 
-Note: Similarly, the default API key can be passed in as an environment variable OPENAI_API_KEY=sk-xxx during startup using the -e flag.  
-For SSL setup, refer to the above. Environment variable configuration can be found [here](README_EN.md#configuration).
+Note: For SSL setup, refer to the above. Environment variable configuration can be found [here](README_EN.md#configuration).
 
 ## Source Code Deployment
 

diff --git a/openai_forward/__init__.py b/openai_forward/__init__.py
@@ -1,4 +1,4 @@
-__version__ = "0.7.0"
+__version__ = "0.7.1"
 
 from dotenv import load_dotenv
 

diff --git a/openai_forward/content/openai.py b/openai_forward/content/openai.py
@@ -1,5 +1,4 @@
 import os
-import pickle
 import threading
 import time
 from abc import ABC, abstractmethod