Skip to content

Commit

Permalink
fix bug when only one reference is provided
Browse files Browse the repository at this point in the history
  • Loading branch information
rudongyu committed Aug 28, 2024
1 parent dec0f78 commit ab0d535
Show file tree
Hide file tree
Showing 3 changed files with 11 additions and 4 deletions.
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,8 @@ ragchecker-cli \
--checker_name=bedrock/meta.llama3-70b-instruct-v1:0 \
--batch_size_extractor=64 \
--batch_size_checker=64 \
--metrics all_metrics
--metrics all_metrics \
# --disable_joint_check # uncomment this line for one-by-one checking, slower but slightly more accurate
```

Please refer to [RefChecker's guidance](https://github.com/amazon-science/RefChecker/tree/main?tab=readme-ov-file#choose-models-for-the-extractor-and-checker) for setting up the extractor and checker models.
Expand Down
4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "ragchecker"
version = "0.1.1"
version = "0.1.2"
description = "RAGChecker: A Fine-grained Framework For Diagnosing Retrieval-Augmented Generation (RAG) systems."
authors = [
"Xiangkun Hu <xiangkhu@amazon.com>",
Expand All @@ -15,7 +15,7 @@ license = "Apache-2.0"

[tool.poetry.dependencies]
python = "^3.9"
refchecker = "^0.2.2"
refchecker = "^0.2.3"
loguru = "^0.7"
dataclasses-json = "^0.6"

Expand Down
8 changes: 7 additions & 1 deletion ragchecker/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,11 @@ def get_args():
parser.add_argument(
'--openai_api_key', type=str
)
parser.add_argument(
"--disable_joint_check", action="store_false", dest="joint_check",
help="Disable joint checking of the claims."
)
parser.set_defaults(joint_check=True)

return parser.parse_args()

Expand All @@ -68,7 +73,8 @@ def main():
checker_api_base=args.checker_api_base,
batch_size_extractor=args.batch_size_extractor,
batch_size_checker=args.batch_size_checker,
openai_api_key=args.openai_api_key
openai_api_key=args.openai_api_key,
joint_check=args.joint_check,
)
with open(args.input_path, "r") as f:
rag_results = RAGResults.from_json(f.read())
Expand Down

0 comments on commit ab0d535

Please sign in to comment.