Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some models reject multiple inputs #35

Open
Kardbord opened this issue May 2, 2024 · 0 comments
Open

Some models reject multiple inputs #35

Kardbord opened this issue May 2, 2024 · 0 comments
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed

Comments

@Kardbord
Copy link
Owner

Kardbord commented May 2, 2024

Despite the docs indicating that multiple inputs are supported for several request types, some models only support a single input at a time. Support should be added to this library for making requests with either a list of inputs or a single input in order to support models that do not like lists.

Example:

# 1) Does not work
curl https://api-inference.huggingface.co/models/meta-llama/Meta-Llama-3-8B-Instruct  \
    -X POST \
    -d '{"inputs": ["The quick brown fox", "jumps over the lazy dog"]}'  \
    -H "Authorization: Bearer ${HUGGING_FACE_TOKEN}" \
    -H 'Content-Type: application/json'

# 2) Works
curl https://api-inference.huggingface.co/models/meta-llama/Meta-Llama-3-8B-Instruct  \
    -X POST \
    -d '{"inputs": "The quick brown fox"}'  \
    -H "Authorization: Bearer ${HUGGING_FACE_TOKEN}" \
    -H 'Content-Type: application/json'

# 3) Works
curl https://api-inference.huggingface.co/models/gpt2 \
    -X POST  \
    -d '{"inputs": ["The quick brown fox", "jumps over the lazy dog"]}'  \
    -H "Authorization: Bearer ${HUGGING_FACE_TOKEN}" \
    -H 'Content-Type: application/json'

1) returns

Failed to deserialize the JSON body into the target type: inputs: invalid type: sequence, expected a string at line 1 column 11

2) returns

[
  {
    "generated_text": "The quick brown fox jumped over the lazy dog. The lazy dog chased after the quick brown fox. The quick brown fox ran around the corner and the lazy dog followed. The fight began. The quick brown fox, being quick and agile, dodged and weaved around the lazy dog's sloppy strikes. The lazy dog, with its powerful jaws, tried to sink its teeth into the quick brown fox's fast-moving body. The quick brown fox snapped back at the lazy dog with a sharp... Read More\n2-"
  }
]

3) returns

[
  [
    {
      "generated_text": "The quick brown fox outcropping her was an easy and natural fit. She was so fast and agile even those with great balance would have had difficulty taking a step or two. For another of her big hindrances, I was able to pick"
    }
  ],
  [
    {
      "generated_text": "jumps over the lazy dog, but I can't help but notice she is wearing some sort of dress (which is a fitting comparison for this part of town), and her ears are covered with small, bright pink earrings (which are quite fitting"
    }
  ]
]
@Kardbord Kardbord added enhancement New feature or request help wanted Extra attention is needed good first issue Good for newcomers labels May 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant