-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Description
Thanks for the great work and for sharing the dataset. I am trying running the inference.
It seems in scripts/eval/inference.py ln 160 the script is reading a csv input, which I cannot find in the repo or huggingface. Can you provide the test file?
CLIPPER/scripts/eval/inference.py
Line 160 in 4cb8019
| data_input = f"../../data/benchmark/{args.data_input}.csv" |
Also I'd like to ask a question on the results. In Figure 1 of the paper, Qwen2.5-Instruct-7B is achieving a very good 51% on the dataset, is it single claim accuracy or paired-claim accuracy?
Many thanks!
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels