Test Answers in CLEVR

[Victoria2333]

I sent an email with my test file to @jcjohnson, i would like to know how long it will take for reply???
Thank for your attention~~~

[jcjohnson]

We followed up via email.

For others viewing this thread in the future, test-set answers should be formatted as a text file with one answer per line, in the same order as the questions in the JSON file; I will then evaluate using this script:

https://gist.github.com/jcjohnson/5796ff43e67835cb8e1de882a5867994

[Victoria2333]

Thank you, i will modify my file format and send it to you later~~~

[Victoria2333]

@jcjohnson
Hi, do I need to include the other variables such as question_id to know which predicted answer is corresponding with the question?? However,it seems that it is enough to offer predicted answers in json file??And the answers are not sored in list,just need to be written in??
Thank you for attention~

[jcjohnson]

@Victoria2333 As mentioned above, your test-set answer should be a text file with one answer per line; you do not need to include question IDs. Please refer to the script I linked; you can make sure your file format is correct by using my script to evaluate your answers on the validation set.

[Victoria2333]

I have already sent to you,
Looking forward to your reply~

[TimeLighter]

Dear Justin.
I have sent a request for evaluation of the test answer in CLEVR.
Thanks in advance for your help.

[jcjohnson]

@TimeLighter I think I've responded to all pending test-set requests; please let me know if you still need test set results.

[TimeLighter]

I got your email.
Thanks a lot :)

[TimeLighter]

Hi.
I sent one more request with email. (modelC_test_answer.txt)
I'm looking forward your replay.
Thank you.

[cyhbrilliant]

@jcjohnson
Hi Justin:
I have already sent you a email with test_24_18.txt attachment file for evaluation about CLEVR 1.0, could you please replay as soon as possible.
Thank you : ).

[jcjohnson]

@cyhbrilliant I responded via email

[cyhbrilliant]

@jcjohnson
Thanks Justin:
We‘ve seen that, and another two files send to you.
Looking forward your reply, thanks very much.

[yuzcccc]

@jcjohnson
Hi, johnson，
I just send a mail to you to test CLEVR results. Hope to get a results soon. Thanks!

[vmichals]

@jcjohnson Sent an email requesting some test results. Hoping to get them soon. Thanks!

[YL1113]

@jcjohnson
Hi, johnson，
I tried to test my latest VQA system in CLEVR_v1.0 dataset. I just send a mail to you to test CLEVR results. could you please reply me the results as soon as possible. very thanks!
submit_answer.txt

[ArrasL]

Hi Justin,
we sent you an email last week to your umich dot edu address with our test answers. We hope that's the right way to get the test accuracy ;-)
Thanks in advance!

[jcjohnson]

@YL1113 @ArrasL replied over email

[benbogin]

We have just sent you our predictions as well and would appreciate your response, thanks so much!

[cengzy14]

@jcjohnson
Hi, johnson，
I just send a mail to you to test CLEVR results (pred_mymodel_test.txt). Hope to get a result soon. Thanks a lot!
(Sorry for send two emails via different email addresses, you can just reply one. Thanks in advance!

[jcjohnson]

@cengzy14 I replied on email. Good luck on CVPR!

[cengzy14]

@cengzy14 I replied on email. Good luck on CVPR!

got it! thank you for your kindness! : )