Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks

Code for NeurIPS 2021 Paper "Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks" by Hanxun Huang, Yisen Wang, Sarah Monazam Erfani, Quanquan Gu, James Bailey, Xingjun Ma

Robust Configurations for WideResNet (WRN-34-R)

• Model defined in models/RobustWideResNet.py

def RobustWideResNet34(num_classes=10):
    # WRN-34-R configurations
    return RobustWideResNet(
        num_classes=num_classes, channel_configs=[16, 320, 640, 512],
        depth_configs=[5, 5, 5], stride_config=[1, 2, 2], stem_stride=1,
        drop_rate_config=[0.0, 0.0, 0.0], zero_init_residual=False,
        block_types=['basic_block', 'basic_block', 'basic_block'],
        activations=['ReLU', 'ReLU', 'ReLU'], is_imagenet=False,
        use_init=True)

Reproduce results from the paper

• Pretrained Weights for WRN-34-R used in Table 2 available on Google Drive
• All hyperparameters/settings for each model/method used in Table 2 are stored in configs/*.yaml files.

Evaluations of the robustness of WRN-34-R

WRN-34-R trained with TRADES

Replace PGD with other attacks ['CW', 'GAMA', 'AA'].

python main.py --config_path configs/config-WRN-34-R
               --exp_name /path/to/experiments/folders
               --version WRN-34-R-trades
               --load_best_model --attack PGD --data_parallel

WRN-34-R trained with TRADES and additional 500k data

Replace PGD with other attacks ['CW', 'GAMA', 'AA'].

python main.py --config_path configs/config-WRN-34-R
               --exp_name /path/to/experiments/folders
               --version WRN-34-R-trades-500k
               --load_best_model --attack PGD --data_parallel

Train WRN-34-R with 500k additional data from scratch

python main.py --config_path configs/config-WRN-34-R
               --exp_name /path/to/experiments/folders
               --version WRN-34-R-trades-500k
               --train --data_parallel

CIFAR-10 - Linf AutoAttack Leaderboard using additional 500k data

• Note: This is not maintained, please find up-to-date leaderboard is available in RobustBench.

Citation

@inproceedings{huang2021exploring,
    title={Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks},
    author={Hanxun Huang and Yisen Wang and Sarah Monazam Erfani and Quanquan Gu and James Bailey and Xingjun Ma},
    booktitle={NeurIPS},
    year={2021}
}

Part of the code is based on the following repo:

• MART: https://github.com/YisenWang/MART
• TREADES: https://github.com/yaodongyu/TRADES
• RST: https://github.com/yaircarmon/semisup-adv
• AutoAttack: https://github.com/fra31/auto-attack
• GAMA: https://github.com/val-iisc/GAMA-GAT