-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reproducing results #5
Comments
Hi, can you show which CLIP model you are using? For different architectures, the result might be slightly different. |
Hi, ViT-B/32 - I'm just running clip_coco.py. Thanks. |
Hi, I just take a look. The recall results are evaluated with the original captions. This should have been mentioned in the experimental details. Sorry for the confusion. |
Hi, as I mentioned, I can't reproduce the bias results as well. Thank you very much for replying. |
Hi @Faldict ! thanks for the great work. I would also appreciate if you could provide all these input files so that it is easier for us to replicate! thanks 👍 |
Hi, I'm having problems while trying to reproduce the numeric results using your code.
For CLIP I'm getting
R1:29.66
R5:54.14
R10:65.24
, which is different but very close to the results presented in the paper.But for CLIP-clip I'm getting
R1:23.82
R5:46.5
R10:56.56
, which is much worse.Thanks.
The text was updated successfully, but these errors were encountered: