Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prompt ${xx_prompt} failed to parse output: The output parser failed to parse the output including retries. #1856

Open
hsx1612727380 opened this issue Jan 20, 2025 · 1 comment
Labels
bug Something isn't working question Further information is requested

Comments

@hsx1612727380
Copy link

[ ] I checked the documentation and related resources and couldn't find an answer to my question.

Your Question
what is unclear to you? What would you like to know?
When I perform Rag evaluation, I have the following issues:

  1. Prompt context_precision_prompt failed to parse output
  2. Prompt claim_decomposition_prompt failed to parse output
  3. Prompt n_l_i_statement_prompt failed to parse output
  4. Prompt extract_entities_prompt failed to parse output
  5. Prompt statement_generator_prompt failed to parse output
    May I ask how to solve this

Code Examples
This community speaks code. Share your code snippets to help us understand your question better.

Additional context
Anything else you want to share with us?

@hsx1612727380 hsx1612727380 added the question Further information is requested label Jan 20, 2025
@dosubot dosubot bot added the bug Something isn't working label Jan 20, 2025
@Manon-56
Copy link

Manon-56 commented Jan 20, 2025

I get the same error when running evaluate with context_precision and context_recall : "Prompt fix_output_format failed to parse output" and "Prompt context_recall_classification_prompt failed to parse output". The response given by the LLM is quite unstable because with the same input data and script, this won't always occur: on rare occasions, the evaluation goes through the entire dataset without raising any error. Maybe increasing the number of retries could help solve the problem

Ragas version 0.2.11

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants