Code to reproduce the results in the paper "Tensor Component Analysis for Interpreting the Latent Space of GANs".

Overview

Tensor Component Analysis for Interpreting the Latent Space of GANs

[ paper | project page ]

Code to reproduce the results in the paper "Tensor Component Analysis for Interpreting the Latent Space of GANs".

./images/teaser.png

dependencies

Firstly, to install the required packages, please run:

$ pip install -r requirements.txt

Pretrained weights

To replicate the results in the paper, you'll need to first download the pre-trained weights. To do so, simply run this from the command line:

./download_weights.sh

Quantitative results

building the prediction matrices

To reproduce Fig. 5, one can then run the ./quant.ipynb notebook using the pre-computed classification scores (please see this notebook for more details).

manually computing predictions

To call the Microsoft Azure Face API to generate the predictions again from scratch, one can run the shell script in ./quant/classify.sh. Firstly however, you need to generate our synthetic images to classify, which we detail below.

Qualitative results

generating the images

Reproducing the qualitative results (i.e. in Fig. 6) involves generating synthetic faces and 3 edited versions with the 3 attributes of interest (hair colour, yaw, and pitch). To generate these images (which are also used for the quantitative results), simply run:

$ ./generate_quant_edits.sh

mode-wise edits

./images/116-blonde.gif ./images/116-yaw.gif ./images/116-pitch.gif

Manual edits along individual modes of the tensor are made by calling main.py with the --mode edit_modewise flag. For example, one can reproduce the images from Fig. 3 with:

$ python main.py --cp_rank 0 --tucker_ranks "4,4,4,512" --model_name pggan_celebahq1024 --penalty_lam 0.001 --resume_iters 1000
  --n_to_edit 10 \
  --mode edit_modewise \
  --attribute_to_edit male

multilinear edits

./images/thick.gif

Edits achieved with the 'multilinear mixing' are achieved instead by loading the relevant weights and supplying the --mode edit_multilinear flag. For example, the images in Fig. 4 are generated with:

$ python main.py --cp_rank 0 --tucker_ranks "256,4,4,512" --model_name pggan_celebahq1024 --penalty_lam 0.001 --resume_iters 200000
  --n_to_edit 10 \
  --mode edit_multilinear \
  --attribute_to_edit thick

Please feel free to get in touch at: [email protected], where x=oldfield


credits

All the code in ./architectures/ and utils.py is directly imported from https://github.com/genforce/genforce, only lightly modified to support performing the forward pass through the models partially, and returning the intermediate tensors.

The structure of the codebase follows https://github.com/yunjey/stargan, and hence we use their code as a template to build off. For this reason, you will find small helper functions (e.g. the first few lines of main.py) are borrowed from the StarGAN codebase.

Owner
James Oldfield
James Oldfield
UniMoCo: Unsupervised, Semi-Supervised and Full-Supervised Visual Representation Learning

UniMoCo: Unsupervised, Semi-Supervised and Full-Supervised Visual Representation Learning This is the official PyTorch implementation for UniMoCo pape

dddzg 49 Jan 02, 2023
The source code of the ICCV2021 paper "PIRenderer: Controllable Portrait Image Generation via Semantic Neural Rendering"

Website | ArXiv | Get Start | Video PIRenderer The source code of the ICCV2021 paper "PIRenderer: Controllable Portrait Image Generation via Semantic

Ren Yurui 261 Jan 09, 2023
PyTorch Lightning implementation of Automatic Speech Recognition

lasr Lightening Automatic Speech Recognition An MIT License ASR research library, built on PyTorch-Lightning, for developing end-to-end ASR models. In

Soohwan Kim 40 Sep 19, 2022
Improving Compound Activity Classification via Deep Transfer and Representation Learning

Improving Compound Activity Classification via Deep Transfer and Representation Learning This repository is the official implementation of Improving C

NingLab 2 Nov 24, 2021
Serve TensorFlow ML models with TF-Serving and then create a Streamlit UI to use them

TensorFlow Serving + Streamlit! ✨ 🖼️ Serve TensorFlow ML models with TF-Serving and then create a Streamlit UI to use them! This is a pretty simple S

Álvaro Bartolomé 18 Jan 07, 2023
CAPRI: Context-Aware Interpretable Point-of-Interest Recommendation Framework

CAPRI: Context-Aware Interpretable Point-of-Interest Recommendation Framework This repository contains a framework for Recommender Systems (RecSys), a

RecSys Lab 8 Jul 03, 2022
PyTorch implementation of MoCo v3 for self-supervised ResNet and ViT.

MoCo v3 for Self-supervised ResNet and ViT Introduction This is a PyTorch implementation of MoCo v3 for self-supervised ResNet and ViT. The original M

Facebook Research 887 Jan 08, 2023
Implementing DeepMind's Fast Reinforcement Learning paper

Fast Reinforcement Learning This is a repo where I implement the algorithms in the paper, Fast reinforcement learning with generalized policy updates.

Marcus Chiam 6 Nov 28, 2022
Simulation-based inference for the Galactic Center Excess

Simulation-based inference for the Galactic Center Excess Siddharth Mishra-Sharma and Kyle Cranmer Abstract The nature of the Fermi gamma-ray Galactic

Siddharth Mishra-Sharma 3 Jan 21, 2022
MODALS: Modality-agnostic Automated Data Augmentation in the Latent Space

Update (20 Jan 2020): MODALS on text data is avialable MODALS MODALS: Modality-agnostic Automated Data Augmentation in the Latent Space Table of Conte

38 Dec 15, 2022
Official Pytorch implementation for 2021 ICCV paper "Learning Motion Priors for 4D Human Body Capture in 3D Scenes" and trained models / data

Learning Motion Priors for 4D Human Body Capture in 3D Scenes (LEMO) Official Pytorch implementation for 2021 ICCV (oral) paper "Learning Motion Prior

165 Dec 19, 2022
Asynchronous Advantage Actor-Critic in PyTorch

Asynchronous Advantage Actor-Critic in PyTorch This is PyTorch implementation of A3C as described in Asynchronous Methods for Deep Reinforcement Learn

Reiji Hatsugai 38 Dec 12, 2022
Kaggle Lyft Motion Prediction for Autonomous Vehicles 4th place solution

Lyft Motion Prediction for Autonomous Vehicles Code for the 4th place solution of Lyft Motion Prediction for Autonomous Vehicles on Kaggle. Discussion

44 Jun 27, 2022
This is a simple framework to make object detection dataset very quickly

FastAnnotation Table of contents General info Requirements Setup General info This is a simple framework to make object detection dataset very quickly

Serena Tetart 1 Jan 24, 2022
robomimic: A Modular Framework for Robot Learning from Demonstration

robomimic [Homepage]   [Documentation]   [Study Paper]   [Study Website]   [ARISE Initiative] Latest Updates [08/09/2021] v0.1.0: Initial code and pap

ARISE Initiative 178 Jan 05, 2023
In this project we use both Resnet and Self-attention layer for cat, dog and flower classification.

cdf_att_classification classes = {0: 'cat', 1: 'dog', 2: 'flower'} In this project we use both Resnet and Self-attention layer for cdf-Classification.

3 Nov 23, 2022
A PyTorch implementation of "DGC-Net: Dense Geometric Correspondence Network"

DGC-Net: Dense Geometric Correspondence Network This is a PyTorch implementation of our work "DGC-Net: Dense Geometric Correspondence Network" TL;DR A

191 Dec 16, 2022
Distributional Sliced-Wasserstein distance code

Distributional Sliced Wasserstein distance This is a pytorch implementation of the paper "Distributional Sliced-Wasserstein and Applications to Genera

VinAI Research 39 Jan 01, 2023
An implementation of the paper "A Neural Algorithm of Artistic Style"

A Neural Algorithm of Artistic Style implementation - Neural Style Transfer This is an implementation of the research paper "A Neural Algorithm of Art

Srijarko Roy 27 Sep 20, 2022