This is an extention of "Greedy Gradient Ensemble for Robust Visual Question Answering" (ICCV 2021, Oral). The prerequisites can refer to GGE.
If you find this repo helpful, please cite
@article{han2021general,
title={General Greedy De-bias Learning},
author={Han, Xinzhe and Wang, Shuhui and Su, Chi and Huang, Qingming and Tian, Qi},
journal={arXiv preprint arXiv:2112.10572 },
year={2021}
}
@inproceedings{han2021greedy,
title={Greedy gradient ensemble for robust visual question answering},
author={Han, Xinzhe and Wang, Shuhui and Su, Chi and Huang, Qingming and Tian, Qi},
booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
pages={1584--1593},
year={2021}
}
The core of GGD is to decompose the CE loss of GGE. The pytorch implementation is
loss = -(logits.log_softmax(-1) * labels).mean() + weight * (logits.log_softmax(-1) * bias * labels).mean()
Biased-MNIST is produced the same with Re-bias. Run the notebook in the MNIST folder. Modify data_label_correlation for different bias level.
Codes for VQA-CP is updated in GGE. To train a model with new GGD-CR, set import base_model_sfce in mian.py and run
CUDA_VISIBLE_DEVICES=0 python main.py --dataset cpv2 --mode gge_iter --debias ce_decompose --topq 1 --topv -1 --qvp 5 --output []
The experiment for AdQA is provided in squad folder. The code is modified from squad.
Some codes are modified from CSS, UpDn, squad, and Re-bias. Thanks for their open source.