Unable to reproduce results

[ritwickchaudhry]

Hi,
Thanks a lot for your implementation. I tried running it on the VizWiz data with the default hyperparameters in the repo. Unfortunately, the model only achieves around 36% accuracy on the test set, with around 49% on the train set, and hence suffers from a lot of overfitting. Can you please tell me if there are some other things to finetune or why that might be happening?
I'm using the data from here: https://vizwiz.org/tasks-and-datasets/vqa/