Skip to content

Commit

Permalink
minor edit
Browse files Browse the repository at this point in the history
Signed-off-by: SumanthRH <sumanthrh@anyscale.com>
  • Loading branch information
SumanthRH committed Oct 25, 2024
1 parent b82244d commit 4b367f7
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 2 deletions.
2 changes: 1 addition & 1 deletion templates/e2e-dspy-workflow/README.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -1668,7 +1668,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Serving a DSPy Pipeline as a Ray Serve Application\n",
"## Serving a DSPy Pipeline using Ray Serve\n",
"\n",
"We can break down our program into two distinct parts: 1) Fine-tuned LLM served behind an OpenAI compatible endpoint and 2) The DSPy program (our business logic tying all components together)\n",
"\n",
Expand Down
2 changes: 1 addition & 1 deletion templates/e2e-dspy-workflow/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -976,7 +976,7 @@ To deploy, you can serve the optimized DSPy program directly: This is the simple

NOTE: As of DSPy 2.5, there are scalability limitations for high throughput scenarios with DSPy. DSPy compiled programs currently use threading for handling multiple queries in parallel, which might not scale as well as a native `async` implementation. A native `async` implementation is in the immediate roadmap for DSPy. If this is a concern, you can always try to stitch together the saved program from DSPy in native Python code.

## Serving a DSPy Pipeline as a Ray Serve Application
## Serving a DSPy Pipeline using Ray Serve

We can break down our program into two distinct parts: 1) Fine-tuned LLM served behind an OpenAI compatible endpoint and 2) The DSPy program (our business logic tying all components together)

Expand Down

0 comments on commit 4b367f7

Please sign in to comment.