Multitask Learning Strengthens Adversarial Robustness

Related tags

Deep LearningMTRobust
Overview

Multitask Learning Strengthens Adversarial Robustness

@inproceedings{mao2020multitask,
  author    = {Chengzhi Mao and
               Amogh Gupta and
               Vikram Nitin and
               Baishakhi Ray and
               Shuran Song and
               Junfeng Yang and
               Carl Vondrick},
  title     = {Multitask Learning Strengthens Adversarial Robustness},
  booktitle = {Computer Vision - {ECCV} 2020 - 16th European Conference, Glasgow,
               UK, August 23-28, 2020, Proceedings, Part {II}},
  series    = {Lecture Notes in Computer Science},
  volume    = {12347},
  pages     = {158--174},
  publisher = {Springer},
  year      = {2020},
  url       = {https://doi.org/10.1007/978-3-030-58536-5\_10},
  doi       = {10.1007/978-3-030-58536-5\_10},
}

Demo for Robustness under multitask attack

Download Cityscapes dataset from Cityscapes.

Download pretrained DRN-22 model from DRN model zoo.

Modify the path to data and model in demo_mtlrobust.py.

Run demo to see the trend that model overall robustness is increased when the output dimension increased.

To see the gradient norm measurement of robustness, set get_grad=True,

To see the actually robust accuracy for model, set test_acc_output_dim=False

python demo_mtlrobust.py

which explains why segmentation is inherently robust.

CityScape

Data preprocessing

Run python data_resize_cityscape.py to resize to smaller images.

Train Robust model against single task attack

  1. Set up the path to data in config/drn_d_22_cityscape_config.json

  2. Run cityscape_example.sh to train a main task with auxiliary task for robustness.

Taskonomy

Data Preprocessing

You can use our preprocessed data from preprocessed data

Or do from scratch

  1. Download data from official raw data.

  2. Run python data_resize_taskonomy.py to resize to smaller images.

  3. Rename segment_semantic to segmentsemantic.

Train Robust model against single task attack

  1. Set up the path to data in config/resnet18_taskonomy_config.json

  2. Run taskonomy_example.sh to train a main task with auxiliary task for robustness. For different task, we have different different setup, refer to our paper and supplementary for details.

Model evaluation

We offer our pretrained models to download here: Cityscapes segmentation depth and Taskonomy taskonomy segmentation demo

After setting up the path to your downloaded models in test_cityscapes_seg.py and test_taskonomy_seg.py,

Run python test_cityscapes_seg.py and python test_taskonomy_seg.py for evaluating the robustness of multitask models under single task attacks.

Pretrained models for other tasks for Taskonomy can be downloaded [here, comming soon](comming soon)

Acknowledgement

Our code refer the code at: https://github.com/fyu/drn/blob/master/drn.py Taskonomy https://github.com/tstandley/taskgrouping,

We thank the authors for open sourcing their code.

Owner
Columbia University
Columbia University
A curated list of awesome resources related to Semantic Search🔎 and Semantic Similarity tasks.

A curated list of awesome resources related to Semantic Search🔎 and Semantic Similarity tasks.

224 Jan 04, 2023
BMVC 2021 Oral: code for BI-GCN: Boundary-Aware Input-Dependent Graph Convolution for Biomedical Image Segmentation

BMVC 2021 BI-GConv: Boundary-Aware Input-Dependent Graph Convolution for Biomedical Image Segmentation Necassary Dependencies: PyTorch 1.2.0 Python 3.

Yanda Meng 15 Nov 08, 2022
FastFace: Lightweight Face Detection Framework

Light Face Detection using PyTorch Lightning

Ömer BORHAN 75 Dec 05, 2022
The undersampled DWI image using Slice-Interleaved Diffusion Encoding (SIDE) method can be reconstructed by the UNet network.

UNet-SIDE The undersampled DWI image using Slice-Interleaved Diffusion Encoding (SIDE) method can be reconstructed by the UNet network. For Super Reso

TIANTIAN XU 1 Jan 13, 2022
NHL 94 AI contests

nhl94-ai The end goals of this project is to: Train Models that play NHL 94 Support AI vs AI contests in NHL 94 Provide an improved AI opponent for NH

Mathieu Poliquin 2 Dec 06, 2021
Autonomous Ground Vehicle Navigation and Control Simulation Examples in Python

Autonomous Ground Vehicle Navigation and Control Simulation Examples in Python THIS PROJECT IS CURRENTLY A WORK IN PROGRESS AND THUS THIS REPOSITORY I

Joshua Marshall 14 Dec 31, 2022
Data, model training, and evaluation code for "PubTables-1M: Towards a universal dataset and metrics for training and evaluating table extraction models".

PubTables-1M This repository contains training and evaluation code for the paper "PubTables-1M: Towards a universal dataset and metrics for training a

Microsoft 365 Jan 04, 2023
Accelerated Multi-Modal MR Imaging with Transformers

Accelerated Multi-Modal MR Imaging with Transformers Dependencies numpy==1.18.5 scikit_image==0.16.2 torchvision==0.8.1 torch==1.7.0 runstats==1.8.0 p

54 Dec 16, 2022
Official Code for VideoLT: Large-scale Long-tailed Video Recognition (ICCV 2021)

Pytorch Code for VideoLT [Website][Paper] Updates [10/29/2021] Features uploaded to Google Drive, for access please send us an e-mail: zhangxing18 at

Skye 26 Sep 18, 2022
Code for the paper Learning the Predictability of the Future

Learning the Predictability of the Future Code from the paper Learning the Predictability of the Future. Website of the project in hyperfuture.cs.colu

Computer Vision Lab at Columbia University 139 Nov 18, 2022
Advanced yabai wooting scripts

Yabai Wooting scripts Installation requirements Both https://github.com/xiamaz/python-yabai-client and https://github.com/xiamaz/python-wooting-rgb ne

Max Zhao 3 Dec 31, 2021
Python based Advanced AI Assistant

Knick is a virtual artificial intelligence project, fully developed in python. The objective of this project is to develop a virtual assistant that can handle our minor, intermediate as well as heavy

19 Nov 15, 2022
PyoMyo - Python Opensource Myo library

PyoMyo Python module for the Thalmic Labs Myo armband. Cross platform and multithreaded and works without the Myo SDK. pip install pyomyo Documentati

PerlinWarp 81 Jan 08, 2023
Code for "R-GCN: The R Could Stand for Random"

RR-GCN: Random Relational Graph Convolutional Networks PyTorch Geometric code for the paper "R-GCN: The R Could Stand for Random" RR-GCN is an extensi

PreDiCT.IDLab 31 Sep 07, 2022
Explainability for Vision Transformers (in PyTorch)

Explainability for Vision Transformers (in PyTorch) This repository implements methods for explainability in Vision Transformers

Jacob Gildenblat 442 Jan 04, 2023
Txt2Xml tool will help you convert from txt COCO format to VOC xml format in Object Detection Problem.

TXT 2 XML All codes assume running from root directory. Please update the sys path at the beginning of the codes before running. Over View Txt2Xml too

Nguyễn Trường Lâu 4 Nov 24, 2022
simple demo codes for Learning to Teach with Dynamic Loss Functions

Learning to Teach with Dynamic Loss Functions This repo contains the simple demo for the NeurIPS-18 paper: Learning to Teach with Dynamic Loss Functio

Lijun Wu 15 Dec 30, 2021
Supervised multi-SNE (S-multi-SNE): Multi-view visualisation and classification

S-multi-SNE Supervised multi-SNE (S-multi-SNE): Multi-view visualisation and classification A repository containing the code to reproduce the findings

Theodoulos Rodosthenous 3 Apr 15, 2022
Neural Oblivious Decision Ensembles

Neural Oblivious Decision Ensembles A supplementary code for anonymous ICLR 2020 submission. What does it do? It learns deep ensembles of oblivious di

25 Sep 21, 2022
[NeurIPS 2021] SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learning

SSUL - Official Pytorch Implementation (NeurIPS 2021) SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learning Sun

Clova AI Research 44 Dec 27, 2022