Add support for other LLM providers

[amirhmoradi]

I would like to be able to use any model from OpenAI, Anthropic, Cohere, Forefront, HuggingFace, Aleph Alpha, Replicate, Banana and llama.cpp and more.

Some projects that provide this feature:

• https://github.com/nat/openplayground
• https://github.com/okisdev/ChatChat

I would love to contribute if your provide me with some help on how to get started :)

[molander]

+1!
We can start by just honoring anything with:
OPENAI_API_BASE=<some_other_server>
or perhaps passing the proxy option could work:
OPENAI_API_PROXY=<somewhere_else:/v1/completions>

Great project btw! Kudos and thank you for open-sourcing! Since it is opensource, it does indeed make sense to support some other models, especially opensource ones!

[krrishdholakia]

Hey - we might be able to help - https://github.com/BerriAI/litellm/tree/main/cookbook/proxy-server

Why use a proxy server instead of calling them directly though? With a server you'd need to run it somewhere and add an extra hop (latency)