Code for the CVPR2022 paper "Frequency-driven Imperceptible Adversarial Attack on Semantic Similarity"

Overview

Introduction

This is an official release of the paper "Frequency-driven Imperceptible Adversarial Attack on Semantic Similarity" (arxiv link). Overview

Abstract: Current adversarial attack research reveals the vulnerability of learning-based classifiers against carefully crafted perturbations. However, most existing attack methods have inherent limitations in cross-dataset generalization as they rely on a classification layer with a closed set of categories. Furthermore, the perturbations generated by these methods may appear in regions easily perceptible to the human visual system (HVS). To circumvent the former problem, we propose a novel algorithm that attacks semantic similarity on feature representations. In this way, we are able to fool classifiers without limiting attacks to a specific dataset. For imperceptibility, we introduce the low-frequency constraint to limit perturbations within high-frequency components, ensuring perceptual similarity between adversarial examples and originals. Extensive experiments on three datasets(CIFAR-10, CIFAR-100, and ImageNet-1K) and three public online platforms indicate that our attack can yield misleading and transferable adversarial examples across architectures and datasets. Additionally, visualization results and quantitative performance (in terms of four different metrics) show that the proposed algorithm generates more imperceptible perturbations than the state-of-the-art methods. Our code will be publicly available.

Requirements

  • python ==3.6
  • torch == 1.7.0
  • torchvision >= 0.7
  • numpy == 1.19.2
  • Pillow == 8.0.1
  • pywt

Required Dataset

  1. The data structure of Cifar10, Cifar100, ImageNet or any other datasets look like below. Please modify the dataloader at SSAH-Adversarial-master/main.py/ accordingly for your dataset structure.
/dataset/
├── Cifar10
│   │   ├── cifar-10-python.tar.gz
├── Cifar-100-python
│   │   ├── cifar-100-python.tar.gz
├── imagenet
│   ├── val
│   │   ├── n02328150

Experiments

We trained a resnet20 model with 92.6% accuracy with CIFAR1010 and a resnet20 model with 69.63% accuracy with CIFAR100. If you want to have a test, you can download our pre-trained models with the Google Drivers. If you want to use our algorithm to attack your own trained model, you can always replace our models in the file checkpoints.

(1)Attack the Models Trained on Cifar10

CUDA_VISIBLE_DEVICES=0,1 bash scripts/cifar/cifar10-r20.sh

(2)Attack the Models Trained on Cifar100

CUDA_VISIBLE_DEVICES=0,1 bash scripts/cifar/cifar100-r20.sh

(2)Attack the Models Trained on Imagenet_val

CUDA_VISIBLE_DEVICES=0,1 bash scripts/cifar/Imagenet_val-r50.sh

Examples

example

Results on CIFAR10 Here we offer some experiment results. You can get more results in our paper.

Name Knowledge ASR(%) L2 Linf FID LF Paper
BIM White Box 100.0 0.85 0.03 14.85 0.25 ICLR2017
PGD White Box 100.0 1.28 0.03 27.86 0.34 arxiv link
MIM White Box 100.0 1.90 0.03 26.00 0.48 CVPR2018
AutoAttack White Box 100.0 1.91 0.03 34.93 0.61 ICML2020
AdvDrop White Box 99.92 0.90 0.07 16.34 0.34 ICCV2021
C&W White Box 100.0 0.39 0.06 8.23 0.11 IEEE SSP2017
PerC-AL White Box 98.29 0.86 0.18 9.58 0.15 CVPR2020
SSA White Box 99.96 0.29 0.02 5.73 0.07 CVPR2022
SSAH White Box 99.94 0.26 0.02 5.03 0.03 CVPR2022

Citation

if the code or method help you in the research, please cite the following paper:

@article{luo2022frequency,
  title={Frequency-driven Imperceptible Adversarial Attack on Semantic Similarity},
  author={Luo, Cheng and Lin, Qinliang and Xie, Weicheng and Wu, Bizhu and Xie, Jinheng and Shen, Linlin},
  journal={arXiv preprint arXiv:2203.05151},
  year={2022}
}
Issues
  • The variable adv was reassigned after iteration, which seems to be an error

    The variable adv was reassigned after iteration, which seems to be an error

    def forward(self, inputs: torch.Tensor) -> torch.Tensor:
    
        with torch.no_grad():
            inputs_fea = self.fea_extract(self.normalize_fn(inputs))
    
        # low frequency component
        inputs_ll = self.DWT(inputs)
        inputs_ll = self.IDWT(inputs_ll)
    
        # changes of variables
        eps = 3e-7
        modifier = torch.arctanh(inputs * (2 - eps * 2) - 1 + eps)
        modifier = Variable(modifier, requires_grad=True)
        modifier = modifier.to(self.device)
        optimizer = optim.Adam([modifier], lr=self.lr)
    
        lowFre_loss = nn.SmoothL1Loss(reduction='sum')
    
        for step in range(self.num_iteration):
            optimizer.zero_grad()
            self.encoder_fea.zero_grad()
    
            adv = 0.5 * (torch.tanh(modifier) + 1)
            adv_fea = self.fea_extract(self.normalize_fn(adv))
    
            adv_ll = self.DWT(adv)
            adv_ll = self.IDWT(adv_ll)
    
            pos_sim, neg_sim = self.cal_sim(adv_fea, inputs_fea)
            if step == 0:
                pos_neg_sim, indices = self.select_setp1(pos_sim, neg_sim)
            else:
                pos_neg_sim = self.select_step2(pos_sim, neg_sim, indices)
    
            sim_pos = pos_neg_sim[:, 0]
            sim_neg = pos_neg_sim[:, -1]
    
            w_p = torch.clamp_min(sim_pos.detach() - self.m, min=0)
            w_n = torch.clamp_min(1 + self.m - sim_neg.detach(), min=0)
    
            adv_cost = self.alpha * torch.sum(torch.clamp(w_p * sim_pos - w_n * sim_neg, min=0))
            lowFre_cost = self.beta * lowFre_loss(adv_ll, inputs_ll)
            total_cost = adv_cost + lowFre_cost
    
            optimizer.zero_grad()
            total_cost.backward()
            optimizer.step()
    
        adv = 0.5 * (torch.tanh(modifier.detach()) + 1)
        return adv
    
    opened by SuperCarryFu 0
The LaTeX and Python code for generating the paper, experiments' results and visualizations reported in each paper is available (whenever possible) in the paper's directory

This repository contains the software implementation of most algorithms used or developed in my research. The LaTeX and Python code for generating the

João Fonseca 2 Apr 2, 2022
Inference code for "StylePeople: A Generative Model of Fullbody Human Avatars" paper. This code is for the part of the paper describing video-based avatars.

NeuralTextures This is repository with inference code for paper "StylePeople: A Generative Model of Fullbody Human Avatars" (CVPR21). This code is for

Visual Understanding Lab @ Samsung AI Center Moscow 18 Apr 13, 2022
This is the official source code for SLATE. We provide the code for the model, the training code, and a dataset loader for the 3D Shapes dataset. This code is implemented in Pytorch.

SLATE This is the official source code for SLATE. We provide the code for the model, the training code and a dataset loader for the 3D Shapes dataset.

Gautam Singh 41 Mar 29, 2022
Code for paper ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop.

Who Left the Dogs Out? Evaluation and demo code for our ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization

Benjamin Biggs 25 Apr 5, 2022
TensorFlow code for the neural network presented in the paper: "Structural Language Models of Code" (ICML'2020)

SLM: Structural Language Models of Code This is an official implementation of the model described in: "Structural Language Models of Code" [PDF] To ap

null 66 Mar 30, 2022
Code for the prototype tool in our paper "CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoning".

CoProtector Code for the prototype tool in our paper "CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoning".

Zhensu Sun 1 Oct 26, 2021
Code to use Augmented Shapiro Wilks Stopping, as well as code for the paper "Statistically Signifigant Stopping of Neural Network Training"

This codebase is being actively maintained, please create and issue if you have issues using it Basics All data files are included under losses and ea

J K Terry 32 Nov 9, 2021
Code for our method RePRI for Few-Shot Segmentation. Paper at http://arxiv.org/abs/2012.06166

Region Proportion Regularized Inference (RePRI) for Few-Shot Segmentation In this repo, we provide the code for our paper : "Few-Shot Segmentation Wit

Malik Boudiaf 122 Apr 1, 2022
Code for ACM MM 2020 paper "NOH-NMS: Improving Pedestrian Detection by Nearby Objects Hallucination"

NOH-NMS: Improving Pedestrian Detection by Nearby Objects Hallucination The offical implementation for the "NOH-NMS: Improving Pedestrian Detection by

Tencent YouTu Research 55 Apr 7, 2022
Official TensorFlow code for the forthcoming paper

~ Efficient-CapsNet ~ Are you tired of over inflated and overused convolutional neural networks? You're right! It's time for CAPSULES :)

Vittorio Mazzia 177 Apr 15, 2022
This is the code for the paper "Contrastive Clustering" (AAAI 2021)

Contrastive Clustering (CC) This is the code for the paper "Contrastive Clustering" (AAAI 2021) Dependency python>=3.7 pytorch>=1.6.0 torchvision>=0.8

Yunfan Li 168 Apr 16, 2022
Code for the paper Learning the Predictability of the Future

Learning the Predictability of the Future Code from the paper Learning the Predictability of the Future. Website of the project in hyperfuture.cs.colu

Computer Vision Lab at Columbia University 130 Apr 20, 2022
PyTorch code for the paper: FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning

FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning This is the PyTorch implementation of our paper: FeatMatch: Feature-Based Augmentat

null 40 Feb 3, 2022
Code for the paper A Theoretical Analysis of the Repetition Problem in Text Generation

A Theoretical Analysis of the Repetition Problem in Text Generation This repository share the code for the paper "A Theoretical Analysis of the Repeti

Zihao Fu 35 Feb 26, 2022
Code for our ICASSP 2021 paper: SA-Net: Shuffle Attention for Deep Convolutional Neural Networks

SA-Net: Shuffle Attention for Deep Convolutional Neural Networks (paper) By Qing-Long Zhang and Yu-Bin Yang [State Key Laboratory for Novel Software T

Qing-Long Zhang 157 Apr 15, 2022
Open source repository for the code accompanying the paper 'Non-Rigid Neural Radiance Fields Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video'.

Non-Rigid Neural Radiance Fields This is the official repository for the project "Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synt

Facebook Research 221 Apr 17, 2022
Code for the Shortformer model, from the paper by Ofir Press, Noah A. Smith and Mike Lewis.

Shortformer This repository contains the code and the final checkpoint of the Shortformer model. This file explains how to run our experiments on the

Ofir Press 138 Apr 15, 2022
PyTorch code for ICLR 2021 paper Unbiased Teacher for Semi-Supervised Object Detection

Unbiased Teacher for Semi-Supervised Object Detection This is the PyTorch implementation of our paper: Unbiased Teacher for Semi-Supervised Object Detection

Facebook Research 311 Apr 1, 2022
Official code for paper "Optimization for Oriented Object Detection via Representation Invariance Loss".

Optimization for Oriented Object Detection via Representation Invariance Loss By Qi Ming, Zhiqiang Zhou, Lingjuan Miao, Xue Yang, and Yunpeng Dong. Th

ming71 51 Apr 22, 2022