Official pytorch code for "APP: Anytime Progressive Pruning"

Last update: Nov 22, 2022

Overview

APP: Anytime Progressive Pruning

Diganta Misra^1,2,3, Bharat Runwal^2,4, Tianlong Chen⁵, Zhangyang Wang⁵, Irina Rish^1,3

¹ Mila - Quebec AI Institute,² Landskape AI,³ UdeM,⁴ IIT-Delhi,⁵ VITA, UT-Austin

Requirements

To create a new conda environment with the dependencies used in this project, do:

conda env create -f app.yml

For running the code on Restricted-Imagenet Dataset, first install the robustness library from here and provide the imagenet_path argument as the path to the imaganet data folder.

Run the Code

Here is an example of running the Anytime Progressive Pruning (APP) on Cifar-10 dataset with 8 megabatches in total:

python main_anytime_train.py \
    --data ../data \
    --dataset cifar10 \
    --arch resnet50 \
    --seed 1 \
    --epochs 50 \
    --decreasing_lr 20,40 \
    --batch_size 64 \
    --weight_decay 1e-4 \
    --meta_batch_size 6250 \
    --meta_batch_number 8 \
    --sparsity_level 4.5 \
    --snip_size 0.20 \
    --save_dir c10_r50

One-Shot pruning :

python main_anytime_one.py \
    --data ../data \
    --dataset cifar10 \
    --arch resnet50 \
    --seed 1 \
    --epochs 50 \
    --decreasing_lr 20,40 \
    --batch_size 64 \
    --weight_decay 1e-4 \
    --meta_batch_size 6250 \
    --meta_batch_number 8 \
    --sparsity_level 4.5 \
    --snip_size 0.20 \
    --save_dir c10_OSP_r18

Baseline :

python main_anytime_baseline.py \
    --data ../data \
    --dataset cifar10 \
    --arch resnet50 \
    --seed 1 \
    --epochs 50 \
    --decreasing_lr 20,40 \
    --batch_size 64 \
    --weight_decay 1e-4 \
    --meta_batch_size 6250 \
    --meta_batch_number 8 \
    --save_dir c10_BASE_r50

Cite:

@misc{misra2022app,
    title={APP: Anytime Progressive Pruning},
    author={Diganta Misra and Bharat Runwal and Tianlong Chen and Zhangyang Wang and Irina Rish},
    year={2022},
    eprint={2204.01640},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

Official pytorch code for "APP: Anytime Progressive Pruning"

Related tags

Overview

APP: Anytime Progressive Pruning

Requirements

Run the Code

Cite:

Owner

Landskape AI

Code for "Diversity can be Transferred: Output Diversification for White- and Black-box Attacks"

This is a collection of our NAS and Vision Transformer work.

Dual Attention Network for Scene Segmentation (CVPR2019)

Exploit ILP to learn symmetry breaking constraints of ASP programs.

Code for Ditto: Building Digital Twins of Articulated Objects from Interaction

Codes for AAAI 2022 paper: Context-aware Health Event Prediction via Transition Functions on Dynamic Disease Graphs

RefineGNN - Iterative refinement graph neural network for antibody sequence-structure co-design (RefineGNN)

Companion code for the paper Theoretical characterization of uncertainty in high-dimensional linear classification

Official code repository for A Simple Long-Tailed Rocognition Baseline via Vision-Language Model.

Hippocampal segmentation using the UNet network for each axis

The Noise Contrastive Estimation for softmax output written in Pytorch

Revisiting Self-Training for Few-Shot Learning of Language Model.

Source code for Acorn, the precision farming rover by Twisted Fields

A time series processing library

Hypersearch weight debugging and losses tutorial

This is the official implementation of "One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval".

Head2Toe: Utilizing Intermediate Representations for Better OOD Generalization

Kaggle-titanic - A tutorial for Kaggle's Titanic: Machine Learning from Disaster competition. Demonstrates basic data munging, analysis, and visualization techniques. Shows examples of supervised machine learning techniques.

Pytorch Implementation of "Contrastive Representation Learning for Exemplar-Guided Paraphrase Generation"

StableSims is an open-source project aimed at simulating MakerDAO's Dai stablecoin system