diff --git a/docs/source/_static/img/examples/comfyui/add-comfy-pack-input-node.gif b/docs/source/_static/img/examples/comfyui/add-comfy-pack-input-node.gif new file mode 100644 index 00000000000..ecaecd67a05 Binary files /dev/null and b/docs/source/_static/img/examples/comfyui/add-comfy-pack-input-node.gif differ diff --git a/docs/source/_static/img/examples/comfyui/add-comfy-pack-output-node.gif b/docs/source/_static/img/examples/comfyui/add-comfy-pack-output-node.gif new file mode 100644 index 00000000000..82a3a248695 Binary files /dev/null and b/docs/source/_static/img/examples/comfyui/add-comfy-pack-output-node.gif differ diff --git a/docs/source/_static/img/examples/comfyui/install-comfy-pack-via-comfyui-manager.png b/docs/source/_static/img/examples/comfyui/install-comfy-pack-via-comfyui-manager.png new file mode 100644 index 00000000000..839667355b8 Binary files /dev/null and b/docs/source/_static/img/examples/comfyui/install-comfy-pack-via-comfyui-manager.png differ diff --git a/docs/source/_static/img/use-cases/diffusion-models/controlnet/controlnet-bentocloud.png b/docs/source/_static/img/examples/controlnet/controlnet-bentocloud.png similarity index 100% rename from docs/source/_static/img/use-cases/diffusion-models/controlnet/controlnet-bentocloud.png rename to docs/source/_static/img/examples/controlnet/controlnet-bentocloud.png diff --git a/docs/source/_static/img/use-cases/diffusion-models/controlnet/example-image.png b/docs/source/_static/img/examples/controlnet/example-image.png similarity index 100% rename from docs/source/_static/img/use-cases/diffusion-models/controlnet/example-image.png rename to docs/source/_static/img/examples/controlnet/example-image.png diff --git a/docs/source/_static/img/use-cases/diffusion-models/controlnet/output-image.png b/docs/source/_static/img/examples/controlnet/output-image.png similarity index 100% rename from docs/source/_static/img/use-cases/diffusion-models/controlnet/output-image.png rename to docs/source/_static/img/examples/controlnet/output-image.png diff --git a/docs/source/_static/img/use-cases/diffusion-models/controlnet/service-ui.png b/docs/source/_static/img/examples/controlnet/service-ui.png similarity index 100% rename from docs/source/_static/img/use-cases/diffusion-models/controlnet/service-ui.png rename to docs/source/_static/img/examples/controlnet/service-ui.png diff --git a/docs/source/_static/img/use-cases/large-language-models/function-calling/function-calling-diagram.png b/docs/source/_static/img/examples/function-calling/function-calling-diagram.png similarity index 100% rename from docs/source/_static/img/use-cases/large-language-models/function-calling/function-calling-diagram.png rename to docs/source/_static/img/examples/function-calling/function-calling-diagram.png diff --git a/docs/source/_static/img/use-cases/large-language-models/function-calling/function-calling-playground.gif b/docs/source/_static/img/examples/function-calling/function-calling-playground.gif similarity index 100% rename from docs/source/_static/img/use-cases/large-language-models/function-calling/function-calling-playground.gif rename to docs/source/_static/img/examples/function-calling/function-calling-playground.gif diff --git a/docs/source/_static/img/use-cases/large-language-models/function-calling/function-calling-playground.png b/docs/source/_static/img/examples/function-calling/function-calling-playground.png similarity index 100% rename from docs/source/_static/img/use-cases/large-language-models/function-calling/function-calling-playground.png rename to docs/source/_static/img/examples/function-calling/function-calling-playground.png diff --git a/docs/source/_static/img/use-cases/large-language-models/langgraph/langgraph-agent-on-bentocloud.png b/docs/source/_static/img/examples/langgraph/langgraph-agent-on-bentocloud.png similarity index 100% rename from docs/source/_static/img/use-cases/large-language-models/langgraph/langgraph-agent-on-bentocloud.png rename to docs/source/_static/img/examples/langgraph/langgraph-agent-on-bentocloud.png diff --git a/docs/source/_static/img/use-cases/large-language-models/langgraph/langgraph-bentoml-architecture.png b/docs/source/_static/img/examples/langgraph/langgraph-bentoml-architecture.png similarity index 100% rename from docs/source/_static/img/use-cases/large-language-models/langgraph/langgraph-bentoml-architecture.png rename to docs/source/_static/img/examples/langgraph/langgraph-bentoml-architecture.png diff --git a/docs/source/_static/img/use-cases/custom-models/mlflow/bentocloud-ui.png b/docs/source/_static/img/examples/mlflow/bentocloud-ui.png similarity index 100% rename from docs/source/_static/img/use-cases/custom-models/mlflow/bentocloud-ui.png rename to docs/source/_static/img/examples/mlflow/bentocloud-ui.png diff --git a/docs/source/_static/img/use-cases/custom-models/mlflow/service-ui.png b/docs/source/_static/img/examples/mlflow/service-ui.png similarity index 100% rename from docs/source/_static/img/use-cases/custom-models/mlflow/service-ui.png rename to docs/source/_static/img/examples/mlflow/service-ui.png diff --git a/docs/source/_static/img/use-cases/diffusion-models/sdxl-turbo/output-image.png b/docs/source/_static/img/examples/sdxl-turbo/output-image.png similarity index 100% rename from docs/source/_static/img/use-cases/diffusion-models/sdxl-turbo/output-image.png rename to docs/source/_static/img/examples/sdxl-turbo/output-image.png diff --git a/docs/source/_static/img/use-cases/diffusion-models/sdxl-turbo/sdxl-turbo-bentocloud.png b/docs/source/_static/img/examples/sdxl-turbo/sdxl-turbo-bentocloud.png similarity index 100% rename from docs/source/_static/img/use-cases/diffusion-models/sdxl-turbo/sdxl-turbo-bentocloud.png rename to docs/source/_static/img/examples/sdxl-turbo/sdxl-turbo-bentocloud.png diff --git a/docs/source/_static/img/use-cases/diffusion-models/sdxl-turbo/service-ui.png b/docs/source/_static/img/examples/sdxl-turbo/service-ui.png similarity index 100% rename from docs/source/_static/img/use-cases/diffusion-models/sdxl-turbo/service-ui.png rename to docs/source/_static/img/examples/sdxl-turbo/service-ui.png diff --git a/docs/source/_static/img/use-cases/large-language-models/shieldgemma/architecture-shield.png b/docs/source/_static/img/examples/shieldgemma/architecture-shield.png similarity index 100% rename from docs/source/_static/img/use-cases/large-language-models/shieldgemma/architecture-shield.png rename to docs/source/_static/img/examples/shieldgemma/architecture-shield.png diff --git a/docs/source/_static/img/use-cases/large-language-models/shieldgemma/shieldgemma-bentocloud.png b/docs/source/_static/img/examples/shieldgemma/shieldgemma-bentocloud.png similarity index 100% rename from docs/source/_static/img/use-cases/large-language-models/shieldgemma/shieldgemma-bentocloud.png rename to docs/source/_static/img/examples/shieldgemma/shieldgemma-bentocloud.png diff --git a/docs/source/_static/img/use-cases/large-language-models/vllm/service-ui.png b/docs/source/_static/img/examples/vllm/service-ui.png similarity index 100% rename from docs/source/_static/img/use-cases/large-language-models/vllm/service-ui.png rename to docs/source/_static/img/examples/vllm/service-ui.png diff --git a/docs/source/_static/img/use-cases/large-language-models/vllm/vllm-bentocloud.png b/docs/source/_static/img/examples/vllm/vllm-bentocloud.png similarity index 100% rename from docs/source/_static/img/use-cases/large-language-models/vllm/vllm-bentocloud.png rename to docs/source/_static/img/examples/vllm/vllm-bentocloud.png diff --git a/docs/source/_static/img/use-cases/custom-models/xgboost/bentocloud-ui.png b/docs/source/_static/img/examples/xgboost/bentocloud-ui.png similarity index 100% rename from docs/source/_static/img/use-cases/custom-models/xgboost/bentocloud-ui.png rename to docs/source/_static/img/examples/xgboost/bentocloud-ui.png diff --git a/docs/source/_static/img/use-cases/custom-models/xgboost/service-ui.png b/docs/source/_static/img/examples/xgboost/service-ui.png similarity index 100% rename from docs/source/_static/img/use-cases/custom-models/xgboost/service-ui.png rename to docs/source/_static/img/examples/xgboost/service-ui.png diff --git a/docs/source/_static/img/use-cases/audio/whisperx/service-ui.png b/docs/source/_static/img/use-cases/audio/whisperx/service-ui.png deleted file mode 100644 index 90d6a416163..00000000000 Binary files a/docs/source/_static/img/use-cases/audio/whisperx/service-ui.png and /dev/null differ diff --git a/docs/source/_static/img/use-cases/audio/whisperx/whisperx-bentocloud.png b/docs/source/_static/img/use-cases/audio/whisperx/whisperx-bentocloud.png deleted file mode 100644 index fb2d57f3873..00000000000 Binary files a/docs/source/_static/img/use-cases/audio/whisperx/whisperx-bentocloud.png and /dev/null differ diff --git a/docs/source/_static/img/use-cases/audio/xtts/service-ui.png b/docs/source/_static/img/use-cases/audio/xtts/service-ui.png deleted file mode 100644 index eb0180952cf..00000000000 Binary files a/docs/source/_static/img/use-cases/audio/xtts/service-ui.png and /dev/null differ diff --git a/docs/source/_static/img/use-cases/audio/xtts/xtts-bentocloud.png b/docs/source/_static/img/use-cases/audio/xtts/xtts-bentocloud.png deleted file mode 100644 index 1df76c8805b..00000000000 Binary files a/docs/source/_static/img/use-cases/audio/xtts/xtts-bentocloud.png and /dev/null differ diff --git a/docs/source/_static/img/use-cases/blip/blip-bentocloud.png b/docs/source/_static/img/use-cases/blip/blip-bentocloud.png deleted file mode 100644 index 1ac66f5c6ef..00000000000 Binary files a/docs/source/_static/img/use-cases/blip/blip-bentocloud.png and /dev/null differ diff --git a/docs/source/_static/img/use-cases/blip/service-ui.png b/docs/source/_static/img/use-cases/blip/service-ui.png deleted file mode 100644 index e7df071eff0..00000000000 Binary files a/docs/source/_static/img/use-cases/blip/service-ui.png and /dev/null differ diff --git a/docs/source/_static/img/use-cases/diffusion-models/comfyui/add_input_node.png b/docs/source/_static/img/use-cases/diffusion-models/comfyui/add_input_node.png deleted file mode 100644 index 8c9f9159572..00000000000 Binary files a/docs/source/_static/img/use-cases/diffusion-models/comfyui/add_input_node.png and /dev/null differ diff --git a/docs/source/_static/img/use-cases/diffusion-models/comfyui/convert_widget.png b/docs/source/_static/img/use-cases/diffusion-models/comfyui/convert_widget.png deleted file mode 100644 index 8a209146b16..00000000000 Binary files a/docs/source/_static/img/use-cases/diffusion-models/comfyui/convert_widget.png and /dev/null differ diff --git a/docs/source/_static/img/use-cases/diffusion-models/sdxl-lcm-lora/output-image.png b/docs/source/_static/img/use-cases/diffusion-models/sdxl-lcm-lora/output-image.png deleted file mode 100644 index 00af41472e4..00000000000 Binary files a/docs/source/_static/img/use-cases/diffusion-models/sdxl-lcm-lora/output-image.png and /dev/null differ diff --git a/docs/source/_static/img/use-cases/diffusion-models/sdxl-lcm-lora/sdxl-lcm-bentocloud.png b/docs/source/_static/img/use-cases/diffusion-models/sdxl-lcm-lora/sdxl-lcm-bentocloud.png deleted file mode 100644 index 868a30d819a..00000000000 Binary files a/docs/source/_static/img/use-cases/diffusion-models/sdxl-lcm-lora/sdxl-lcm-bentocloud.png and /dev/null differ diff --git a/docs/source/_static/img/use-cases/diffusion-models/sdxl-lcm-lora/service-ui.png b/docs/source/_static/img/use-cases/diffusion-models/sdxl-lcm-lora/service-ui.png deleted file mode 100644 index 299bac78081..00000000000 Binary files a/docs/source/_static/img/use-cases/diffusion-models/sdxl-lcm-lora/service-ui.png and /dev/null differ diff --git a/docs/source/_static/img/use-cases/diffusion-models/svd/girl-image-output.gif b/docs/source/_static/img/use-cases/diffusion-models/svd/girl-image-output.gif deleted file mode 100644 index 0bd5a95da30..00000000000 Binary files a/docs/source/_static/img/use-cases/diffusion-models/svd/girl-image-output.gif and /dev/null differ diff --git a/docs/source/_static/img/use-cases/diffusion-models/svd/service-ui.png b/docs/source/_static/img/use-cases/diffusion-models/svd/service-ui.png deleted file mode 100644 index f0594a341fb..00000000000 Binary files a/docs/source/_static/img/use-cases/diffusion-models/svd/service-ui.png and /dev/null differ diff --git a/docs/source/_static/img/use-cases/diffusion-models/svd/svd-bentocloud.png b/docs/source/_static/img/use-cases/diffusion-models/svd/svd-bentocloud.png deleted file mode 100644 index 9f40ee7d09e..00000000000 Binary files a/docs/source/_static/img/use-cases/diffusion-models/svd/svd-bentocloud.png and /dev/null differ diff --git a/docs/source/_static/img/use-cases/embeddings/clip-embeddings/clip-bentocloud.png b/docs/source/_static/img/use-cases/embeddings/clip-embeddings/clip-bentocloud.png deleted file mode 100644 index aece9f6938e..00000000000 Binary files a/docs/source/_static/img/use-cases/embeddings/clip-embeddings/clip-bentocloud.png and /dev/null differ diff --git a/docs/source/_static/img/use-cases/embeddings/clip-embeddings/service-ui.png b/docs/source/_static/img/use-cases/embeddings/clip-embeddings/service-ui.png deleted file mode 100644 index 493e0512fef..00000000000 Binary files a/docs/source/_static/img/use-cases/embeddings/clip-embeddings/service-ui.png and /dev/null differ diff --git a/docs/source/_static/img/use-cases/embeddings/sentence-embeddings/sentence-embedding-bentocloud.png b/docs/source/_static/img/use-cases/embeddings/sentence-embeddings/sentence-embedding-bentocloud.png deleted file mode 100644 index 4c676225fe2..00000000000 Binary files a/docs/source/_static/img/use-cases/embeddings/sentence-embeddings/sentence-embedding-bentocloud.png and /dev/null differ diff --git a/docs/source/_static/img/use-cases/embeddings/sentence-embeddings/service-ui.png b/docs/source/_static/img/use-cases/embeddings/sentence-embeddings/service-ui.png deleted file mode 100644 index 00acfd56914..00000000000 Binary files a/docs/source/_static/img/use-cases/embeddings/sentence-embeddings/service-ui.png and /dev/null differ diff --git a/docs/source/examples/comfyui.rst b/docs/source/examples/comfyui.rst index 9f641fa18e5..32f24cb594e 100644 --- a/docs/source/examples/comfyui.rst +++ b/docs/source/examples/comfyui.rst @@ -1,9 +1,172 @@ -======= -ComfyUI -======= +================================= +ComfyUI: Deploy workflows as APIs +================================= -`ComfyUI `_ is a powerful tool for designing advanced diffusion pipelines. However, once the pipelines are built, deploying and serving them as API endpoints can be challenging and not very straightforward. +`ComfyUI `_ is a powerful tool for designing advanced diffusion workflows. It provides an extensive collection of resources, including shared workflows and custom nodes, to help creative workers and developers generate content without dealing with complex code. However, deploying and serving these workflows as scalable API endpoints can be `complex and non-intuitive `_. -Recognizing the complexity of ComfyUI, BentoML provides a non-intrusive solution to serve existing ComfyUI pipelines as APIs without requiring any pipeline rewrites. It also offers the flexibility to customize the API endpoint's schema and logic. +To address the deployment challenges, the BentoML team developed `comfy-pack `_, a comprehensive toolkit that transforms ComfyUI workflows into production-grade APIs. Specifically, comfy-pack enables you to: -For detailed information, see the `comfy-pack repository `_ and the `release blog post `_. +- Define standardized API schemas for workflow inputs and outputs +- Serve workflows as HTTP endpoints accessible via standard API clients +- Deploy workflows to BentoCloud with enterprise-grade features such as fast autoscaling and built-in observability +- Package the complete workspace as portable artifacts for consistent reproduction + +Installation +------------ + +You can install comfy-pack using either `ComfyUI Manager `_ or Git. + +.. tab-set:: + + .. tab-item:: ComfyUI Manager (Recommended) + + 1. Open **ComfyUI Manager**. + 2. Search for ``comfy-pack`` and click **Install**. + + .. image:: ../../_static/img/examples/comfyui/install-comfy-pack-via-comfyui-manager.png + :alt: Install comfy-pack via ComfyUI Manager + + 3. Click **Restart** and refresh your ComfyUI interface to apply changes. + + .. tab-item:: Git + + Clone the repository into your ComfyUI custom nodes directory: + + .. code-block:: bash + + cd ComfyUI/custom_nodes + git clone https://github.com/bentoml/comfy-pack.git + +Specify input and output nodes +------------------------------ + +When serving ComfyUI workflows as APIs, one key challenge is establishing a standardized schema for workflow inputs and outputs. comfy-pack addresses this by providing dedicated interface nodes that integrate seamlessly with existing workflows without affecting their core functionality. + +1. Right-click a node containing the widget you want to expose. +2. Select **Convert Widget to Input**, then choose the widget name. +3. To add a comfy-pack input node: + + a. Right-click anywhere on the blank space. + + b. Navigate to **Add Node > ComfyPack > input**, then select the desired input node type: + + - **Image Input**: Accepts image type input, similar to the official ``LoadImage`` node. + - **String Input**: Accepts string type input (e.g., prompts). + - **Int Input**: Accepts integer type input (e.g., dimensions, seeds). + - **File Input**: Accepts file type input. + - **Any Input**: Accepts combo type and other input (e.g., custom nodes). + +4. Connect the comfy-pack input node to the widget you converted previously. + + .. image:: ../../_static/img/examples/comfyui/add-comfy-pack-input-node.gif + :alt: Add comfy-pack input node + +5. To add a comfy-pack output node: + + a. Right-click anywhere on the blank space. + + b. Navigate to **Add Node > ComfyPack > output**, then select the desired output node type: + + - **File Output**: Outputs a file path as a string and saves the file to the specified location. + - **Image Output**: Outputs an image, similar to the official ``SaveImage`` node. + +6. Connect the workflow's output to the comfy-pack output node. + + .. image:: ../../_static/img/examples/comfyui/add-comfy-pack-output-node.gif + :alt: Add comfy-pack output node + +7. Run the workflow to ensure it functions as expected. + +Serve workflows as APIs +----------------------- + +You can expose ComfyUI workflows as HTTP APIs that can be called from any client. + +1. On the toolbar at the top of the screen, click **Serve**. +2. Set the desired port (default: ``3000``). +3. Click **Start** to launch the server. The API will be available at ``http://127.0.0.1:``. +4. The server exposes a ``/generate`` endpoint. Use it to submit requests with parameters configured through comfy-pack nodes (e.g., ``prompt``, ``width``, ``height``, ``seed``). For example: + + .. tab-set:: + + .. tab-item:: CURL + + .. code-block:: bash + + curl -X 'POST' \ + 'http://127.0.0.1:3000/generate' \ + -H 'accept: application/octet-stream' \ + -H 'Content-Type: application/json' \ + --output output.png \ + -d '{ + "prompt": "rocks in a bottle", + "width": 512, + "height": 512, + "seed": 1 + }' + + .. tab-item:: Python client + + comfy-pack uses BentoML as its serving framework, allowing you to use the :doc:`BentoML Python client ` for interaction: + + .. code-block:: python + + import bentoml + + with bentoml.SyncHTTPClient("http://127.0.0.1:3000") as client: + result = client.generate( + prompt="rocks in a bottle", + width=512, + height=512, + seed=1 + ) + + .. important:: + + Parameter names in API calls must match your comfy-pack node names. + +Deploy to BentoCloud +-------------------- + +You can deploy your ComfyUI workflow to BentoCloud for better management and scalability. + +1. On the toolbar at the top of the screen, click **Deploy**. +2. In the dialog that appears, set a name and select required models and system packages. +3. Enter your BentoCloud access token. If you don't have a BentoCloud account, `sign up for free `_ and :doc:`create a token `. +4. Click **Push to Cloud** and wait for your Bento to be built. +5. Once it's ready, click **Deploy Now** to open the Bento details page on BentoCloud. +6. Deploy the Bento from the BentoCloud console. + +Package and restore a workspace +------------------------------- + +You can package a ComfyUI workspace into a portable artifact, ensuring it can be easily transferred and unpacked elsewhere with consistent behavior. + +Create a package +^^^^^^^^^^^^^^^^ + +1. On the toolbar at the top of the screen, click **Package**. +2. Set a name for the package. +3. (Optional) Choose which models to include. Note that only model hashes are stored, not the actual files. This keeps package size minimal while ensuring version accuracy. +4. Click **Pack**. Your browser will automatically download a ``.cpack.zip`` file. + +Restore a workspace +^^^^^^^^^^^^^^^^^^^ + +1. Install comfy-pack CLI: + + .. code-block:: bash + + pip install comfy-pack + +2. Unpack the ``.cpack.zip`` file: + + .. code-block:: bash + + comfy-pack unpack .cpack.zip + +When unpacking, comfy-pack restores the original ComfyUI workspace by performing the following steps: + +1. Prepares a Python virtual environment with the exact packages used in the workflow. +2. Clones the specific ComfyUI version and custom nodes, pinned to the exact versions required by the workflow. +3. Searches for and downloads models from common registries like Hugging Face and Civitai. It uses symbolic links for efficient model sharing (i.e., models are downloaded only once and reused across workflows) and verifies model integrity via hash checking. diff --git a/docs/source/examples/controlnet.rst b/docs/source/examples/controlnet.rst index 1b6f77a16c2..d8541cae4bf 100644 --- a/docs/source/examples/controlnet.rst +++ b/docs/source/examples/controlnet.rst @@ -176,15 +176,15 @@ The server is active at `http://localhost:3000 `_. You ca Visit `http://localhost:3000 `_, scroll down to **Service APIs**, specify the image and parameters, and click **Execute**. - .. image:: ../../_static/img/use-cases/diffusion-models/controlnet/service-ui.png + .. image:: ../../_static/img/examples/controlnet/service-ui.png This is the example image used in the request: -.. image:: ../../_static/img/use-cases/diffusion-models/controlnet/example-image.png +.. image:: ../../_static/img/examples/controlnet/example-image.png Expected output: -.. image:: ../../_static/img/use-cases/diffusion-models/controlnet/output-image.png +.. image:: ../../_static/img/examples/controlnet/output-image.png Deploy to BentoCloud -------------------- @@ -217,7 +217,7 @@ First, specify a configuration YAML file (``bentofile.yaml``) to define the buil Once the Deployment is up and running on BentoCloud, you can access it via the exposed URL. -.. image:: ../../_static/img/use-cases/diffusion-models/controlnet/controlnet-bentocloud.png +.. image:: ../../_static/img/examples/controlnet/controlnet-bentocloud.png .. note:: diff --git a/docs/source/examples/function-calling.rst b/docs/source/examples/function-calling.rst index ac6a65ae2bd..a8a3e639cdc 100644 --- a/docs/source/examples/function-calling.rst +++ b/docs/source/examples/function-calling.rst @@ -39,14 +39,14 @@ The application processes this request and responds by converting USD to CAD usi This example is ready for easy deployment and scaling on BentoCloud. With a single command, you can deploy a production-grade application with fast autoscaling, secure deployment in your cloud, and comprehensive observability. -.. image:: ../../_static/img/use-cases/large-language-models/function-calling/function-calling-playground.gif +.. image:: ../../_static/img/examples/function-calling/function-calling-playground.gif Architecture ------------ This example includes two BentoML Services, a Currency Exchange Assistant and an LLM. The LLM Service exposes an OpenAI-compatible API, so that the Exchange Assistant can call the OpenAI client. Here is the general workflow of this example: -.. image:: ../../_static/img/use-cases/large-language-models/function-calling/function-calling-diagram.png +.. image:: ../../_static/img/examples/function-calling/function-calling-diagram.png 1. A user submits a query to the Exchange Assistant's Query API, which processes the query and forwards it to the LLM to determine the required function and extract parameters. 2. With the extracted parameters, the Query API invokes the identified Exchange Function, which is responsible for the exchange conversion using the specified parameters. @@ -260,7 +260,7 @@ BentoCloud provides fast and scalable infrastructure for building and scaling AI .. tab-item:: BentoCloud Playground - .. image:: ../../_static/img/use-cases/large-language-models/function-calling/function-calling-playground.png + .. image:: ../../_static/img/examples/function-calling/function-calling-playground.png .. tab-item:: Python client diff --git a/docs/source/examples/langgraph.rst b/docs/source/examples/langgraph.rst index a4b32f6e07b..1e39114debe 100644 --- a/docs/source/examples/langgraph.rst +++ b/docs/source/examples/langgraph.rst @@ -39,14 +39,14 @@ Example output: This example is ready for easy deployment and scaling on BentoCloud. You can use either external LLM APIs or deploy an open-source LLM together with the LangGraph agent. With a single command, you get a production-grade application with fast autoscaling, secure deployment in your cloud, and comprehensive observability. -.. image:: ../../_static/img/use-cases/large-language-models/langgraph/langgraph-agent-on-bentocloud.png +.. image:: ../../_static/img/examples/langgraph/langgraph-agent-on-bentocloud.png Architecture ------------ This project consists of two main components: a BentoML Service that serves a LangGraph agent as REST APIs and an LLM that generates text. The LLM can be an external API like Claude 3.5 Sonnet or an open-source model served via BentoML (Mistral 7B in this example). -.. image:: ../../_static/img/use-cases/large-language-models/langgraph/langgraph-bentoml-architecture.png +.. image:: ../../_static/img/examples/langgraph/langgraph-bentoml-architecture.png After a user submits a query, it is processed through the LangGraph agent, which includes: @@ -250,7 +250,7 @@ BentoCloud provides fast and scalable infrastructure for building and scaling AI .. tab-item:: BentoCloud Playground - .. image:: ../../_static/img/use-cases/large-language-models/langgraph/langgraph-agent-on-bentocloud.png + .. image:: ../../_static/img/examples/langgraph/langgraph-agent-on-bentocloud.png .. tab-item:: Python client diff --git a/docs/source/examples/mlflow.rst b/docs/source/examples/mlflow.rst index 8f7d4e85335..2fdf7b36ce7 100644 --- a/docs/source/examples/mlflow.rst +++ b/docs/source/examples/mlflow.rst @@ -159,7 +159,7 @@ The server is active at `http://localhost:3000 `_. You c Visit `http://localhost:3000 `_, scroll down to **Service APIs**, specify the data, and click **Execute**. - .. image:: ../../_static/img/use-cases/custom-models/mlflow/service-ui.png + .. image:: ../../_static/img/examples/mlflow/service-ui.png Deploy to BentoCloud -------------------- @@ -189,7 +189,7 @@ Specify a configuration YAML file (``bentofile.yaml``) to define the build optio Once the Deployment is up and running on BentoCloud, you can access it via the exposed URL. -.. image:: ../../_static/img/use-cases/custom-models/mlflow/bentocloud-ui.png +.. image:: ../../_static/img/examples/mlflow/bentocloud-ui.png .. note:: diff --git a/docs/source/examples/sdxl-turbo.rst b/docs/source/examples/sdxl-turbo.rst index 6b987bab10f..a7627069634 100644 --- a/docs/source/examples/sdxl-turbo.rst +++ b/docs/source/examples/sdxl-turbo.rst @@ -135,11 +135,11 @@ The server is active at `http://localhost:3000 `_. You ca Visit `http://localhost:3000 `_, scroll down to **Service APIs**, specify the parameters, and click **Execute**. - .. image:: ../../_static/img/use-cases/diffusion-models/sdxl-turbo/service-ui.png + .. image:: ../../_static/img/examples/sdxl-turbo/service-ui.png Expected output: -.. image:: ../../_static/img/use-cases/diffusion-models/sdxl-turbo/output-image.png +.. image:: ../../_static/img/examples/sdxl-turbo/output-image.png Deploy to BentoCloud -------------------- @@ -168,7 +168,7 @@ First, specify a configuration YAML file (``bentofile.yaml``) to define the buil Once the Deployment is up and running on BentoCloud, you can access it via the exposed URL. -.. image:: ../../_static/img/use-cases/diffusion-models/sdxl-turbo/sdxl-turbo-bentocloud.png +.. image:: ../../_static/img/examples/sdxl-turbo/sdxl-turbo-bentocloud.png .. note:: diff --git a/docs/source/examples/shieldgemma.rst b/docs/source/examples/shieldgemma.rst index 6f7c5ddfeda..5275012f87e 100644 --- a/docs/source/examples/shieldgemma.rst +++ b/docs/source/examples/shieldgemma.rst @@ -40,14 +40,14 @@ It will result in the application raising an exception, indicating the prompt is This example is ready for easy deployment and scaling on BentoCloud. With a single command, you can deploy a production-grade application with fast autoscaling, secure deployment in your cloud, and comprehensive observability. -.. image:: ../../_static/img/use-cases/large-language-models/shieldgemma/shieldgemma-bentocloud.png +.. image:: ../../_static/img/examples/shieldgemma/shieldgemma-bentocloud.png Architecture ------------ This example includes two BentoML Services: ``Gemma`` and ``ShieldAssistant``. ``Gemma`` evaluates the safety of the prompt, and if it is considered safe, ``ShieldAssistant`` proceeds to call the OpenAI GPT-3.5 Turbo API to generate a response. If the probability score from the safety check exceeds a preset threshold, it indicates a potential violation of the safety guidelines. As a result, ``ShieldAssistant`` raises an error and rejects the query. -.. image:: ../../_static/img/use-cases/large-language-models/shieldgemma/architecture-shield.png +.. image:: ../../_static/img/examples/shieldgemma/architecture-shield.png Code explanations ----------------- @@ -211,7 +211,7 @@ BentoCloud provides fast and scalable infrastructure for building and scaling AI .. tab-item:: BentoCloud Playground - .. image:: ../../_static/img/use-cases/large-language-models/shieldgemma/shieldgemma-bentocloud.png + .. image:: ../../_static/img/examples/shieldgemma/shieldgemma-bentocloud.png .. tab-item:: Python client diff --git a/docs/source/examples/vllm.rst b/docs/source/examples/vllm.rst index 75152fbeada..e148fea59f7 100644 --- a/docs/source/examples/vllm.rst +++ b/docs/source/examples/vllm.rst @@ -240,7 +240,7 @@ The server is active at `http://localhost:3000 `_. You ca Visit `http://localhost:3000 `_, scroll down to **Service APIs**, and click **Try it out**. In the **Request body** box, enter your prompt and click **Execute**. - .. image:: ../../_static/img/use-cases/large-language-models/vllm/service-ui.png + .. image:: ../../_static/img/examples/vllm/service-ui.png Deploy to BentoCloud -------------------- @@ -279,7 +279,7 @@ After the Service is ready, you can deploy the project to BentoCloud for better 4. Once the Deployment is up and running on BentoCloud, you can access it via the exposed URL. - .. image:: ../../_static/img/use-cases/large-language-models/vllm/vllm-bentocloud.png + .. image:: ../../_static/img/examples/vllm/vllm-bentocloud.png .. note:: diff --git a/docs/source/examples/xgboost.rst b/docs/source/examples/xgboost.rst index ecdbd1c7efe..16e7f5a3132 100644 --- a/docs/source/examples/xgboost.rst +++ b/docs/source/examples/xgboost.rst @@ -195,7 +195,7 @@ The server is active at `http://localhost:3000 `_. You c Visit `http://localhost:3000 `_, scroll down to **Service APIs**, specify the data, and click **Execute**. - .. image:: ../../_static/img/use-cases/custom-models/xgboost/service-ui.png + .. image:: ../../_static/img/examples/xgboost/service-ui.png Deploy to BentoCloud -------------------- @@ -225,7 +225,7 @@ First, specify a configuration YAML file (``bentofile.yaml``) to define the buil Once the Deployment is up and running on BentoCloud, you can access it via the exposed URL. -.. image:: ../../_static/img/use-cases/custom-models/xgboost/bentocloud-ui.png +.. image:: ../../_static/img/examples/xgboost/bentocloud-ui.png .. note::