Skip to content
This repository has been archived by the owner on Sep 12, 2024. It is now read-only.

Example only using free Hugging Face inference and embeddings #211

Answered by SeeknnDestroy
vanetreg asked this question in Q&A
Discussion options

You must be logged in to vote

Hi @vanetreg, thank you for suggestion! I will definetely update our examples according your suggestion :)

For quick solution, you can set your llm_model and embed_model parameters as Hugging Face models and you are good to go! If you have hosted it on Huggingface, you also should give the llm_api parameter as your hosted url:

tasks:
  - name: "qa"
    llm_model: "huggingface/WizardLM/WizardCoder-Python-34B-V1.0"
    llm_api_base: "https://my-endpoint.huggingface.cloud"
    embed_model: "huggingface/BAAI/bge-large-zh"  

also you might need to set your HUGGINGFACE_API_KEY env variable:

os.environ['HUGGINGFACE_API_KEY'] = ...

Hope it helps. If you have any more suggestions or questions ple…

Replies: 3 comments 1 reply

Comment options

You must be logged in to vote
0 replies
Answer selected by SeeknnDestroy
Comment options

You must be logged in to vote
0 replies
Comment options

You must be logged in to vote
1 reply
@SeeknnDestroy
Comment options

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants
Converted from issue

This discussion was converted from issue #207 on January 14, 2024 22:32.