[NeurIPS2021] Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks

Overview

Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks

Code for NeurIPS 2021 Paper "Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks" by Hanxun Huang, Yisen Wang, Sarah Monazam Erfani, Quanquan Gu, James Bailey, Xingjun Ma


Robust Configurations for WideResNet (WRN-34-R)

def RobustWideResNet34(num_classes=10):
    # WRN-34-R configurations
    return RobustWideResNet(
        num_classes=num_classes, channel_configs=[16, 320, 640, 512],
        depth_configs=[5, 5, 5], stride_config=[1, 2, 2], stem_stride=1,
        drop_rate_config=[0.0, 0.0, 0.0], zero_init_residual=False,
        block_types=['basic_block', 'basic_block', 'basic_block'],
        activations=['ReLU', 'ReLU', 'ReLU'], is_imagenet=False,
        use_init=True)

Reproduce results from the paper

  • Pretrained Weights for WRN-34-R used in Table 2 available on Google Drive
  • All hyperparameters/settings for each model/method used in Table 2 are stored in configs/*.yaml files.

Evaluations of the robustness of WRN-34-R

WRN-34-R trained with TRADES

Replace PGD with other attacks ['CW', 'GAMA', 'AA'].

python main.py --config_path configs/config-WRN-34-R
               --exp_name /path/to/experiments/folders
               --version WRN-34-R-trades
               --load_best_model --attack PGD --data_parallel
WRN-34-R trained with TRADES and additional 500k data

Replace PGD with other attacks ['CW', 'GAMA', 'AA'].

python main.py --config_path configs/config-WRN-34-R
               --exp_name /path/to/experiments/folders
               --version WRN-34-R-trades-500k
               --load_best_model --attack PGD --data_parallel

Train WRN-34-R with 500k additional data from scratch

python main.py --config_path configs/config-WRN-34-R
               --exp_name /path/to/experiments/folders
               --version WRN-34-R-trades-500k
               --train --data_parallel

CIFAR-10 - Linf AutoAttack Leaderboard using additional 500k data

  • Note: This is not maintained, please find up-to-date leaderboard is available in RobustBench.
# paper model architecture clean report. AA
1 (Gowal et al., 2020) available WRN-70-16 91.10 65.87 65.88
2 Ours‡ + EMA available WRN-34-R 91.23 62.54 62.54
3 Ours available WRN-34-R 90.56 61.56 61.56
4 (Wu et al., 2020a) available WRN-34-15 87.67 60.65 60.65
5 (Wu et al., 2020b) available WRN-28-10 88.25 60.04 60.04
6 (Carmon et al., 2019) available WRN-28-10 89.69 62.5 59.53
7 (Sehwag et al., 2020) available WRN-28-10 88.98 - 57.14
8 (Wang et al., 2020) available WRN-28-10 87.50 65.04 56.29

Citation

@inproceedings{huang2021exploring,
    title={Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks},
    author={Hanxun Huang and Yisen Wang and Sarah Monazam Erfani and Quanquan Gu and James Bailey and Xingjun Ma},
    booktitle={NeurIPS},
    year={2021}
}

Part of the code is based on the following repo:

Owner
Hanxun Huang
Hanxun Huang
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators

ELECTRA Introduction ELECTRA is a method for self-supervised language representation learning. It can be used to pre-train transformer networks using

Google Research 2.1k Dec 28, 2022
a pytorch implementation of auto-punctuation learned character by character

Learning Auto-Punctuation by Reading Engadget Articles Link to Other of my work 🌟 Deep Learning Notes: A collection of my notes going from basic mult

Ge Yang 137 Nov 09, 2022
WarpDrive: Extremely Fast End-to-End Deep Multi-Agent Reinforcement Learning on a GPU

WarpDrive is a flexible, lightweight, and easy-to-use open-source reinforcement learning (RL) framework that implements end-to-end multi-agent RL on a single GPU (Graphics Processing Unit).

Salesforce 334 Jan 06, 2023
SOTA model in CIFAR10

A PyTorch Implementation of CIFAR Tricks 调研了CIFAR10数据集上各种trick,数据增强,正则化方法,并进行了实现。目前项目告一段落,如果有更好的想法,或者希望一起维护这个项目可以提issue或者在我的主页找到我的联系方式。 0. Requirement

PJDong 58 Dec 21, 2022
Implementation for "Manga Filling Style Conversion with Screentone Variational Autoencoder" (SIGGRAPH ASIA 2020 issue)

Manga Filling with ScreenVAE SIGGRAPH ASIA 2020 | Project Website | BibTex This repository is for ScreenVAE introduced in the following paper "Manga F

30 Dec 24, 2022
tensorflow implementation of 'YOLO : Real-Time Object Detection'

YOLO_tensorflow (Version 0.3, Last updated :2017.02.21) 1.Introduction This is tensorflow implementation of the YOLO:Real-Time Object Detection It can

Jinyoung Choi 1.7k Nov 21, 2022
🕹️ Official Implementation of Conditional Motion In-betweening (CMIB) 🏃

Conditional Motion In-Betweening (CMIB) Official implementation of paper: Conditional Motion In-betweeening. Paper(arXiv) | Project Page | YouTube in-

Jihoon Kim 81 Dec 22, 2022
This repo provides the official code for TransBTS: Multimodal Brain Tumor Segmentation Using Transformer (https://arxiv.org/pdf/2103.04430.pdf).

TransBTS: Multimodal Brain Tumor Segmentation Using Transformer This repo is the official implementation for TransBTS: Multimodal Brain Tumor Segmenta

Raymond 247 Dec 28, 2022
Codebase for the solution that won first place and was awarded the most human-like agent in the 2021 NeurIPS Competition MineRL BASALT Challenge.

KAIROS MineRL BASALT Codebase for the solution that won first place and was awarded the most human-like agent in the 2021 NeurIPS Competition MineRL B

Vinicius G. Goecks 37 Oct 30, 2022
This is the code repository implementing the paper "TreePartNet: Neural Decomposition of Point Clouds for 3D Tree Reconstruction".

TreePartNet This is the code repository implementing the paper "TreePartNet: Neural Decomposition of Point Clouds for 3D Tree Reconstruction". Depende

刘彦超 34 Nov 30, 2022
Data labels and scripts for fastMRI.org

fastMRI+: Clinical pathology annotations for the fastMRI dataset The fastMRI dataset is a publicly available MRI raw (k-space) dataset. It has been us

Microsoft 51 Dec 22, 2022
Using LSTM to detect spoofing attacks in an Air-Ground network

Using LSTM to detect spoofing attacks in an Air-Ground network Specifications IDE: Spider Packages: Tensorflow 2.1.0 Keras NumPy Scikit-learn Matplotl

Tiep M. H. 1 Nov 20, 2021
PyTorch Implementation of Daft-Exprt: Robust Prosody Transfer Across Speakers for Expressive Speech Synthesis

PyTorch Implementation of Daft-Exprt: Robust Prosody Transfer Across Speakers for Expressive Speech Synthesis

Ubisoft 76 Dec 30, 2022
Official codebase for Pretrained Transformers as Universal Computation Engines.

universal-computation Overview Official codebase for Pretrained Transformers as Universal Computation Engines. Contains demo notebook and scripts to r

Kevin Lu 210 Dec 28, 2022
Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers [CVPR 2021]

Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers [BCNet, CVPR 2021] This is the official pytorch implementation of BCNet built on

Lei Ke 434 Dec 01, 2022
PyTorch implementaton of our CVPR 2021 paper "Bridging the Visual Gap: Wide-Range Image Blending"

Bridging the Visual Gap: Wide-Range Image Blending PyTorch implementaton of our CVPR 2021 paper "Bridging the Visual Gap: Wide-Range Image Blending".

Chia-Ni Lu 69 Dec 20, 2022
Official repository with code and data accompanying the NAACL 2021 paper "Hurdles to Progress in Long-form Question Answering" (https://arxiv.org/abs/2103.06332).

Hurdles to Progress in Long-form Question Answering This repository contains the official scripts and datasets accompanying our NAACL 2021 paper, "Hur

Kalpesh Krishna 41 Nov 08, 2022
constructing maps of intellectual influence from publication data

Influencemap Project @ ANU Influence in the academic communities has been an area of interest for researchers. This can be seen in the popularity of a

CS Metrics 13 Jun 18, 2022
NeuroFind - A solution to the to the Task given by the Oberseminar of Messtechnik Institute of TU Dresden in 2021

NeuroFind A solution to the to the Task given by the Oberseminar of Messtechnik

1 Jan 20, 2022
Semi-Autoregressive Transformer for Image Captioning

Semi-Autoregressive Transformer for Image Captioning Requirements Python 3.6 Pytorch 1.6 Prepare data Please use git clone --recurse-submodules to clo

YE Zhou 23 Dec 09, 2022