Jacobian-Free Backprop

Associated Publication

JFB: Jacobian-Free Backpropagation for Implicit Models (arXiv Link)

@article{WuFung2022JFB,
    title={JFB: Jacobian-Free Backpropagation for Implicit Models},
    author={Fung, Samy Wu and Heaton, Howard and Li, Qiuwei and McKenzie, Daniel and Osher, Stanley and Yin, Wotao},
    journal={Proceedings of the AAAI Conference on Artificial Intelligence},
    year={2022}
}

Set-up

Install all the requirements:

pip install -r requirements.txt

Training

For each dataset, there are three types of training drivers:

FPN with our proposed backprop:

	python train_CIFAR10.py
	python train_CIFAR10_Unaugmented.py
	python train_MNIST.py
	python train_SVHN.py

FPN with Jacobian-based backprop:

	python train_CIFAR10_Jacobian_Based.py
	python train_CIFAR10_Unaugmented_Jacobian_Based.py
	python train_MNIST_Jacobian_Based.py
	python train_SVHN_Jacobian_Based.py

Explicit models.

	python train_CIFAR10_Explicit.py
	python train_CIFAR10_Unaugmented_Explicit.py
	python train_MNIST_Explicit.py
	python train_SVHN_Explicit.py

JFB: Jacobian-Free Backpropagation for Implicit Models

Related tags

Overview

Jacobian-Free Backprop

Associated Publication

Set-up

Training

Owner

Typal Research

Pytorch implementation for Patient Knowledge Distillation for BERT Model Compression

Run Effective Large Batch Contrastive Learning on Limited Memory GPU

Official implementation of the MM'21 paper Constrained Graphic Layout Generation via Latent Optimization

Differentiable Annealed Importance Sampling (DAIS)

Alias-Free Generative Adversarial Networks (StyleGAN3) Official PyTorch implementation

YOLOv5 + ROS2 object detection package

Hand gesture recognition model that can be used as a remote control for a smart tv.

Res2Net for Instance segmentation and Object detection using MaskRCNN

Isaac Gym Reinforcement Learning Environments

Implementation of UNET architecture for Image Segmentation.

The code for 'Deep Residual Fourier Transformation for Single Image Deblurring'

Boosted CVaR Classification (NeurIPS 2021)

Wav2Vec for speech recognition, classification, and audio classification

Code for STFT Transformer used in BirdCLEF 2021 competition.

Research into Forex price prediction from price history using Deep Sequence Modeling with Stacked LSTMs.

High-performance moving least squares material point method (MLS-MPM) solver.

An LSTM based GAN for Human motion synthesis

Similarity-based Gray-box Adversarial Attack Against Deep Face Recognition

Visual odometry package based on hardware-accelerated NVIDIA Elbrus library with world class quality and performance.

Multi-modal Content Creation Model Training Infrastructure including the FACT model (AI Choreographer) implementation.