Code to reproduce the results in the paper "Tensor Component Analysis for Interpreting the Latent Space of GANs".

Overview

Tensor Component Analysis for Interpreting the Latent Space of GANs

[ paper | project page ]

Code to reproduce the results in the paper "Tensor Component Analysis for Interpreting the Latent Space of GANs".

./images/teaser.png

dependencies

Firstly, to install the required packages, please run:

$ pip install -r requirements.txt

Pretrained weights

To replicate the results in the paper, you'll need to first download the pre-trained weights. To do so, simply run this from the command line:

./download_weights.sh

Quantitative results

building the prediction matrices

To reproduce Fig. 5, one can then run the ./quant.ipynb notebook using the pre-computed classification scores (please see this notebook for more details).

manually computing predictions

To call the Microsoft Azure Face API to generate the predictions again from scratch, one can run the shell script in ./quant/classify.sh. Firstly however, you need to generate our synthetic images to classify, which we detail below.

Qualitative results

generating the images

Reproducing the qualitative results (i.e. in Fig. 6) involves generating synthetic faces and 3 edited versions with the 3 attributes of interest (hair colour, yaw, and pitch). To generate these images (which are also used for the quantitative results), simply run:

$ ./generate_quant_edits.sh

mode-wise edits

./images/116-blonde.gif ./images/116-yaw.gif ./images/116-pitch.gif

Manual edits along individual modes of the tensor are made by calling main.py with the --mode edit_modewise flag. For example, one can reproduce the images from Fig. 3 with:

$ python main.py --cp_rank 0 --tucker_ranks "4,4,4,512" --model_name pggan_celebahq1024 --penalty_lam 0.001 --resume_iters 1000
  --n_to_edit 10 \
  --mode edit_modewise \
  --attribute_to_edit male

multilinear edits

./images/thick.gif

Edits achieved with the 'multilinear mixing' are achieved instead by loading the relevant weights and supplying the --mode edit_multilinear flag. For example, the images in Fig. 4 are generated with:

$ python main.py --cp_rank 0 --tucker_ranks "256,4,4,512" --model_name pggan_celebahq1024 --penalty_lam 0.001 --resume_iters 200000
  --n_to_edit 10 \
  --mode edit_multilinear \
  --attribute_to_edit thick

Please feel free to get in touch at: [email protected], where x=oldfield


credits

All the code in ./architectures/ and utils.py is directly imported from https://github.com/genforce/genforce, only lightly modified to support performing the forward pass through the models partially, and returning the intermediate tensors.

The structure of the codebase follows https://github.com/yunjey/stargan, and hence we use their code as a template to build off. For this reason, you will find small helper functions (e.g. the first few lines of main.py) are borrowed from the StarGAN codebase.

Owner
James Oldfield
James Oldfield
AdelaiDet is an open source toolbox for multiple instance-level detection and recognition tasks.

AdelaiDet is an open source toolbox for multiple instance-level detection and recognition tasks.

Adelaide Intelligent Machines (AIM) Group 3k Jan 02, 2023
A Python Library for Graph Outlier Detection (Anomaly Detection)

PyGOD is a Python library for graph outlier detection (anomaly detection). This exciting yet challenging field has many key applications, e.g., detect

PyGOD Team 757 Jan 04, 2023
Segcache: a memory-efficient and scalable in-memory key-value cache for small objects

Segcache: a memory-efficient and scalable in-memory key-value cache for small objects This repo contains the code of Segcache described in the followi

TheSys Group @ CMU CS 78 Jan 07, 2023
Meta-TTS: Meta-Learning for Few-shot SpeakerAdaptive Text-to-Speech

Meta-TTS: Meta-Learning for Few-shot SpeakerAdaptive Text-to-Speech This repository is the official implementation of "Meta-TTS: Meta-Learning for Few

Sung-Feng Huang 128 Dec 25, 2022
Computational modelling of ray propagation through optical elements using the principles of geometric optics (Ray Tracer)

Computational modelling of ray propagation through optical elements using the principles of geometric optics (Ray Tracer) Introduction By applying the

Son Gyo Jung 1 Jul 09, 2022
RARA: Zero-shot Sim2Real Visual Navigation with Following Foreground Cues

RARA: Zero-shot Sim2Real Visual Navigation with Following Foreground Cues FGBG (foreground-background) pytorch package for defining and training model

Klaas Kelchtermans 1 Jun 02, 2022
The official implementation code of "PlantStereo: A Stereo Matching Benchmark for Plant Surface Dense Reconstruction."

PlantStereo This is the official implementation code for the paper "PlantStereo: A Stereo Matching Benchmark for Plant Surface Dense Reconstruction".

Wang Qingyu 14 Nov 28, 2022
Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more

Apache MXNet (incubating) for Deep Learning Apache MXNet is a deep learning framework designed for both efficiency and flexibility. It allows you to m

The Apache Software Foundation 20.2k Jan 05, 2023
Supplementary code for SIGGRAPH 2021 paper: Discovering Diverse Athletic Jumping Strategies

SIGGRAPH 2021: Discovering Diverse Athletic Jumping Strategies project page paper demo video Prerequisites Important Notes We suspect there are bugs i

54 Dec 06, 2022
An Industrial Grade Federated Learning Framework

DOC | Quick Start | 中文 FATE (Federated AI Technology Enabler) is an open-source project initiated by Webank's AI Department to provide a secure comput

Federated AI Ecosystem 4.8k Jan 09, 2023
Improving Object Detection by Label Assignment Distillation

Improving Object Detection by Label Assignment Distillation This is the official implementation of the WACV 2022 paper Improving Object Detection by L

Cybercore Co. Ltd 51 Dec 08, 2022
ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesis

ImageBART NeurIPS 2021 Patrick Esser*, Robin Rombach*, Andreas Blattmann*, Björn Ommer * equal contribution arXiv | BibTeX | Poster Requirements A sui

CompVis Heidelberg 110 Jan 01, 2023
RLMeta is a light-weight flexible framework for Distributed Reinforcement Learning Research.

RLMeta rlmeta - a flexible lightweight research framework for Distributed Reinforcement Learning based on PyTorch and moolib Installation To build fro

Meta Research 281 Dec 22, 2022
Code to reproduce the experiments in the paper "Transformer Based Multi-Source Domain Adaptation" (EMNLP 2020)

Transformer Based Multi-Source Domain Adaptation Dustin Wright and Isabelle Augenstein To appear in EMNLP 2020. Read the preprint: https://arxiv.org/a

CopeNLU 36 Dec 05, 2022
A machine learning library for spiking neural networks. Supports training with both torch and jax pipelines, and deployment to neuromorphic hardware.

Rockpool Rockpool is a Python package for developing signal processing applications with spiking neural networks. Rockpool allows you to build network

SynSense 21 Dec 14, 2022
A TensorFlow implementation of DeepMind's WaveNet paper

A TensorFlow implementation of DeepMind's WaveNet paper This is a TensorFlow implementation of the WaveNet generative neural network architecture for

Igor Babuschkin 5.3k Dec 28, 2022
This repository contains project created during the Data Challenge module at London School of Hygiene & Tropical Medicine

LSHTM_RCS This repository contains project created during the Data Challenge module at London School of Hygiene & Tropical Medicine (LSHTM) in collabo

Lukas Kopecky 3 Jan 30, 2022
Coursera - Quiz & Assignment of Coursera

Coursera Assignments This repository is aimed to help Coursera learners who have difficulties in their learning process. The quiz and programming home

浅梦 828 Jan 04, 2023
Code for database and frontend of webpage for Neural Fields in Visual Computing and Beyond.

Neural Fields in Visual Computing—Complementary Webpage This is based on the amazing MiniConf project from Hendrik Strobelt and Sasha Rush—thank you!

Brown University Visual Computing Group 29 Nov 30, 2022
Official code repository for Continual Learning In Environments With Polynomial Mixing Times

Official code for Continual Learning In Environments With Polynomial Mixing Times Continual Learning in Environments with Polynomial Mixing Times This

Sharath Raparthy 1 Dec 19, 2021