Text Embeddings Inference update

[joaomsimoes]

Hi everyone.

I wanted to use the Text Embeddings Inference with the encoder but I noticed two small bugs in the code. I believe that the HFEndpointEncoder was intentionally created to be used with TEI (right?)

1. The loop for max_retries in attempts, that is inside of the function query, has no break or any system to return the result when we have a success response. I added a break, similar to the OpenAI encoder.

2. The response from TEI is [[[array]]]. The array is inside of a list of a list. I remove one list when receiving the response. Without this it will throw a dimension error when comparing all the vectors.

These are the main bugs, but I would also take some time to purpose a future update. With TEI we can send a batch of texts

curl 127.0.0.1:8080/embed \
    -X POST \
    -d '{"inputs":["Today is a nice day", "I like you"]}' \
    -H 'Content-Type: application/json'

To save time, we could batch the different sentences to the endpoint. This would be great for longer document. If it sounds interesting I can try to help to develop it.

By the way, should I use semantic router for splitting text, or the semantic chunkers?

[joaomsimoes]

Okay, I added the batch to the encoder. The bug number 2 it is now solved.

It is also much faster to batch the requests. Before was taking around a minute for a small document. Now it takes few seconds. I took an arbitrary number for the batch. This can be studied.

[jamescalam]

@joaomsimoes DM'd you on discord but not sure if you still use it - we continued your PR here, can you take a look and let us know if all is correct? Thanks! #423