Pytorch implementation of our paper under review — Lottery Jackpots Exist in Pre-trained Models

Overview

Lottery Jackpots Exist in Pre-trained Models (Paper Link)

Requirements

  • Python >= 3.7.4
  • Pytorch >= 1.6.1
  • Torchvision >= 0.4.1

Reproduce the Experiment Results

  1. Download the pre-trained models from this link and place them in the pre-train folder.

  2. Select a configuration file in configs to reproduce the experiment results reported in the paper. For example, to find a lottery jackpot with 30 epochs for pruning 95% parameters of ResNet-32 on CIFAR-10, run:

    python cifar.py --config configs/resnet32_cifar10/90sparsity30epoch.yaml --gpus 0

    To find a lottery jackpot with 30 epochs for pruning 90% parameters of ResNet-50 on ImageNet, run:

    python imagenet.py --config configs/resnet50_imagenet/90sparsity30epoch.yaml --gpus 0

    Note that the data_path in the yaml file should be changed to the data

Evaluate Our Pruned Models

We provide configuration, training logs, and pruned models reported in the paper. They can be downloaded from the provided links in the following table:

Model Dataset Sparsity Epoch Top-1 Acc. Link
VGGNet-19 CIFAR-10 90% 30 93.88% link
VGGNet-19 CIFAR-10 90% 160 93.94% link
VGGNet-19 CIFAR-10 95% 30 93.49% link
VGGNet-19 CIFAR-10 95% 160 93.74% link
VGGNet-19 CIFAR-100 90% 30 72.59% link
VGGNet-19 CIFAR-100 90% 160 74.61% link
VGGNet-19 CIFAR-100 95% 30 71.76% link
VGGNet-19 CIFAR-100 95% 160 73.35% link
ResNet-32 CIFAR-10 90% 30 93.70% link
ResNet-32 CIFAR-10 90% 160 94.39% link
ResNet-32 CIFAR-10 95% 30 92.90% link
ResNet-32 CIFAR-10 95% 160 93.41% link
ResNet-32 CIFAR-100 90% 30 72.22% link
ResNet-32 CIFAR-100 90% 160 73.43% link
ResNet-32 CIFAR-100 95% 30 69.38% link
ResNet-32 CIFAR-100 95% 160 70.31% link
ResNet-50 ImageNet 80% 30 74.53% link
ResNet-50 ImageNet 80% 60 75.26% link
ResNet-50 ImageNet 90% 30 72.17% link
ResNet-50 ImageNet 90% 60 72.46% link

To test the our pruned models, download the pruned models and place them in the ckpt folder.

  1. Select a configuration file in configs to test the pruned models. For example, to evaluate a lottery jackpot for pruning ResNet-32 on CIFAR-10, run:

    python evaluate.py --config configs/resnet32_cifar10/evaluate.yaml --gpus 0

    To evaluate a lottery jackpot for pruning ResNet-50 on ImageNet, run:

    python evaluate.py --config configs/resnet50_imagenet/evaluate.yaml --gpus 0

Owner
Yuxin Zhang
Deep Neural Network Compression & Acceleration
Yuxin Zhang
Exponential Graph is Provably Efficient for Decentralized Deep Training

Exponential Graph is Provably Efficient for Decentralized Deep Training This code repository is for the paper Exponential Graph is Provably Efficient

3 Apr 20, 2022
IJON is an annotation mechanism that analysts can use to guide fuzzers such as AFL.

IJON SPACE EXPLORER IJON is an annotation mechanism that analysts can use to guide fuzzers such as AFL. Using only a small (usually one line) annotati

Chair for Sys­tems Se­cu­ri­ty 146 Dec 16, 2022
Code for TIP 2017 paper --- Illumination Decomposition for Photograph with Multiple Light Sources.

Illumination_Decomposition Code for TIP 2017 paper --- Illumination Decomposition for Photograph with Multiple Light Sources. This code implements the

QAY 7 Nov 15, 2020
CvT2DistilGPT2 is an encoder-to-decoder model that was developed for chest X-ray report generation.

CvT2DistilGPT2 Improving Chest X-Ray Report Generation by Leveraging Warm-Starting This repository houses the implementation of CvT2DistilGPT2 from [1

The Australian e-Health Research Centre 21 Dec 28, 2022
Code and Data for NeurIPS2021 Paper "A Dataset for Answering Time-Sensitive Questions"

Time-Sensitive-QA The repo contains the dataset and code for NeurIPS2021 (dataset track) paper Time-Sensitive Question Answering dataset. The dataset

wenhu chen 35 Nov 14, 2022
Official NumPy Implementation of Deep Networks from the Principle of Rate Reduction (2021)

Deep Networks from the Principle of Rate Reduction This repository is the official NumPy implementation of the paper Deep Networks from the Principle

Ryan Chan 49 Dec 16, 2022
网络协议2天集训

网络协议2天集训 抓包工具安装 Wireshark wireshark下载地址 Tcpdump CentOS yum install tcpdump -y Ubuntu apt-get install tcpdump -y k8s抓包测试环境 查看虚拟网卡veth pair 查看

120 Dec 12, 2022
Predicting Tweet Sentiment Maching Learning and streamlit

Predicting-Tweet-Sentiment-Maching-Learning-and-streamlit (I prefere using Visual Studio Code ) Open the folder in VS Code Run the first cell in requi

1 Nov 20, 2021
Neural-net-from-scratch - A simple Neural Network from scratch in Python using the Pymathrix library

A Simple Neural Network from scratch A Simple Neural Network from scratch in Pyt

Youssef Chafiqui 2 Jan 07, 2022
Office source code of paper UniFuse: Unidirectional Fusion for 360$^\circ$ Panorama Depth Estimation

UniFuse (RAL+ICRA2021) Office source code of paper UniFuse: Unidirectional Fusion for 360$^\circ$ Panorama Depth Estimation, arXiv, Demo Preparation I

Alibaba 47 Dec 26, 2022
Решения, подсказки, тесты и утилиты для тренировки по алгоритмам от Яндекса.

Решения и подсказки к тренировке по алгоритмам от Яндекса Что есть внутри Решения с подсказками и комментариями; рекомендую сначала смотреть md файл п

Yankovsky Andrey 50 Dec 26, 2022
Code release for SLIP Self-supervision meets Language-Image Pre-training

SLIP: Self-supervision meets Language-Image Pre-training What you can find in this repo: Pre-trained models (with ViT-Small, Base, Large) and code to

Meta Research 621 Dec 31, 2022
[arXiv22] Disentangled Representation Learning for Text-Video Retrieval

Disentangled Representation Learning for Text-Video Retrieval This is a PyTorch implementation of the paper Disentangled Representation Learning for T

Qiang Wang 49 Dec 18, 2022
A GUI for Face Recognition, based upon Docker, Tkinter, GPU and a camera device.

Face Recognition GUI This repository is a GUI version of Face Recognition by Adam Geitgey, where e.g. Docker and Tkinter are utilized. All the materia

Kasper Henriksen 6 Dec 05, 2022
This repository contain code on Novelty-Driven Binary Particle Swarm Optimisation for Truss Optimisation Problems.

This repository contain code on Novelty-Driven Binary Particle Swarm Optimisation for Truss Optimisation Problems. The main directory include the code

0 Dec 23, 2021
Simple and Distributed Machine Learning

Synapse Machine Learning SynapseML (previously MMLSpark) is an open source library to simplify the creation of scalable machine learning pipelines. Sy

Microsoft 3.9k Dec 30, 2022
Official code for "Distributed Deep Learning in Open Collaborations" (NeurIPS 2021)

Distributed Deep Learning in Open Collaborations This repository contains the code for the NeurIPS 2021 paper "Distributed Deep Learning in Open Colla

Yandex Research 96 Sep 15, 2022
Zero-shot Synthesis with Group-Supervised Learning (ICLR 2021 paper)

GSL - Zero-shot Synthesis with Group-Supervised Learning Figure: Zero-shot synthesis performance of our method with different dataset (iLab-20M, RaFD,

Andy_Ge 62 Dec 21, 2022
Implementation of ReSeg using PyTorch

Implementation of ReSeg using PyTorch ReSeg: A Recurrent Neural Network-based Model for Semantic Segmentation Pascal-Part Annotations Pascal VOC 2010

Onur Kaplan 46 Nov 23, 2022
audioLIME: Listenable Explanations Using Source Separation

audioLIME This repository contains the Python package audioLIME, a tool for creating listenable explanations for machine learning models in music info

Institute of Computational Perception 27 Dec 01, 2022