Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Batch encoding for TEI encoder #423

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
+13 −8
Open

feat: Batch encoding for TEI encoder #423

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
8198e13
Text Embeddings Inference update
joaomsimoes Sep 11, 2024
bfce487
batch Text Embeddings Inference
joaomsimoes Sep 12, 2024
3328286
batch Text Embeddings Inference
joaomsimoes Sep 12, 2024
1ed8159
Fixed embeddings list
Vits-99 Sep 20, 2024
79f50b4
Merge branch 'main' into vittorio/tei_update
Vits-99 Sep 20, 2024
2764adf
Fix on embeddings list
Vits-99 Sep 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 4 additions & 6 deletions semantic_router/encoders/huggingface.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -212,19 +212,17 @@ def __call__(self, docs: List[str]) -> List[List[float]]:
ValueError: If no embeddings are returned for a document.
"""

batch_size=50
batch_size = 50
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ValueError: No embeddings returned for batch. Error: Query failed with status 413: {"error":"batch size 50 > maximum allowed batch size 32","error_type":"Validation"}

embeddings = []
for i in range(0, len(docs), batch_size):
batch = docs[i:i + batch_size]
batch = docs[i : i + batch_size]
try:
outputs = self.query({"inputs": batch, "parameters": {}})
if not outputs or len(outputs) == 0:
raise ValueError("No embeddings returned from the query.")
embeddings.extend(outputs)
embeddings = embeddings + outputs
except Exception as e:
raise ValueError(
f"No embeddings returned for batch. Error: {e}"
) from e
raise ValueError(f"No embeddings returned for batch. Error: {e}") from e

return embeddings

Expand Down