InterfaceGAN++: Exploring the limits of InterfaceGAN

Overview

InterfaceGAN++: Exploring the limits of InterfaceGAN

Authors: Apavou Clément & Belkada Younes

Python 3.8 pytorch 1.10.2 sklearn 0.21.2

Open In Colab

From left to right - Images generated using styleGAN and the boundaries Bald, Blond, Heavy_Makeup, Gray_Hair

This the the repository to a project related to the Introduction to Numerical Imaging (i.e, Introduction à l'Imagerie Numérique in French), given by the MVA Masters program at ENS-Paris Saclay. The project and repository is based on the work from Shen et al., and fully supports their codebase. You can refer to the original README) to reproduce their results.

Introduction

In this repository, we propose an approach, termed as InterFaceGAN++, for semantic face editing based on the work from Shen et al. Specifically, we leverage the ideas from the previous work, by applying the method for new face attributes, and also for StyleGAN3. We qualitatively explain that moving the latent vector toward the trained boundaries leads in many cases to keeping the semantic information of the generated images (by preserving its local structure) and modify the desired attribute, thus helps to demonstrate the disentangled property of the styleGANs.

🔥 Additional features

  • Supports StyleGAN2 & StyleGAN3 on the classic attributes
  • New attributes (Bald, Gray hair, Blond hair, Earings, ...) for:
    • StyleGAN
    • StyleGAN2
    • StyleGAN3
  • Supports face generation using StyleGAN3 & StyleGAN2

The list of new features can be found on our attributes detection classifier repository

🔨 Training an attribute detection classifier

We use a ViT-base model to train an attribute detection classifier, please refer to our classification code if you want to test it for new models. Once you retrieve the trained SVM from this repo, you can directly move them in this repo and use them.

Generate images using StyleGAN & StyleGAN2 & StyleGAN3

We did not changed anything to the structure of the old repository, please refer to the previous README. For StyleGAN

🎥 Get the pretrained StyleGAN

We use the styleGAN trained on ffhq for our experiments, if you want to reproduce them, run:

wget -P interfacegan/models/pretrain https://www.dropbox.com/s/qyv37eaobnow7fu/stylegan_ffhq.pth

🎥 Get the pretrained StyleGAN2

We use the styleGAN2 trained on ffhq for our experiments, if you want to reproduce them, run:

wget -P models/pretrain https://api.ngc.nvidia.com/v2/models/nvidia/research/stylegan2/versions/1/files/stylegan2-ffhq-1024x1024.pkl 

🎥 Get the pretrained StyleGAN3

We use the styleGAN3 trained on ffhq for our experiments, if you want to reproduce them, run:

wget -P models/pretrain https://api.ngc.nvidia.com/v2/models/nvidia/research/stylegan3/versions/1/files/stylegan3-t-ffhq-1024x1024.pkl 

The pretrained model should be copied at models/pretrain. If not, move the pretrained model file at this directory.

🎨 Run the generation script

If you want to generate 10 images using styleGAN3 downloaded before, run:

python generate_data.py -m stylegan3_ffhq -o output_stylegan3 -n 10

The arguments are exactly the same as the arguments from the original repository, the code supports the flag -m stylegan3_ffhq for styleGAN3 and -m stylegan3_ffhq for styleGAN2.

✏️ Edit generated images

You can edit the generated images using our trained boundaries! Depending on the generator you want to use, make sure that you have downloaded the right model and put them into models/pretrain.

Examples

Please refer to our interactive google colab notebook to play with our models by clicking the following badge:

Open In Colab

StyleGAN

Example of generated images using StyleGAN and moving the images towards the direction of the attribute grey hair:

original images generated with StyleGAN

grey hair version of the images generated with StyleGAN

StyleGAN2

Example of generated images using StyleGAN2 and moving the images towards the opposite direction of the attribute young:

original images generated with StyleGAN2

non young version of the images generated with StyleGAN2

StyleGAN3

Example of generated images using StyleGAN3 and moving the images towards the attribute beard:

Owner
Younes Belkada
MSc Student in Mathematics - Machine Learning - Perception | M2 MVA @ ENS Paris-Saclay
Younes Belkada
An interactive DNN Model deployed on web that predicts the chance of heart failure for a patient with an accuracy of 98%

Heart Failure Predictor About A Web UI deployed Dense Neural Network Model Made using Tensorflow that predicts whether the patient is healthy or has c

Adit Ahmedabadi 0 Jan 09, 2022
Transfer Learning Remote Sensing

Transfer_Learning_Remote_Sensing Simulation R codes for data generation and visualizations are in the folder simulation. Experiment: California Housin

2 Jun 21, 2022
QRec: A Python Framework for quick implementation of recommender systems (TensorFlow Based)

Introduction QRec is a Python framework for recommender systems (Supported by Python 3.7.4 and Tensorflow 1.14+) in which a number of influential and

Yu 1.4k Dec 30, 2022
Transfer style api - An API to use with Tranfer Style App, where you can use two image and transfer the style

Transfer Style API It's an API to use with Tranfer Style App, where you can use

Brian Alejandro 1 Feb 13, 2022
Library for fast text representation and classification.

fastText fastText is a library for efficient learning of word representations and sentence classification. Table of contents Resources Models Suppleme

Facebook Research 24.1k Jan 01, 2023
The Easy-to-use Dialogue Response Selection Toolkit for Researchers

Easy-to-use toolkit for retrieval-based Chatbot Recent Activity Our released RRS corpus can be found here. Our released BERT-FP post-training checkpoi

GMFTBY 32 Nov 13, 2022
Shape-aware Semi-supervised 3D Semantic Segmentation for Medical Images

SASSnet Code for paper: Shape-aware Semi-supervised 3D Semantic Segmentation for Medical Images(MICCAI 2020) Our code is origin from UA-MT You can fin

klein 125 Jan 03, 2023
Robust Video Matting in PyTorch, TensorFlow, TensorFlow.js, ONNX, CoreML!

Robust Video Matting in PyTorch, TensorFlow, TensorFlow.js, ONNX, CoreML!

Peter Lin 6.5k Jan 04, 2023
Resilience from Diversity: Population-based approach to harden models against adversarial attacks

Resilience from Diversity: Population-based approach to harden models against adversarial attacks Requirements To install requirements: pip install -r

0 Nov 23, 2021
A mini-course offered to Undergrad chemistry students

The best way to use this material is by forking it by click the Fork button at the top, right corner. Then you will get your own copy to play with! Th

Raghu 19 Dec 19, 2022
Implementation for Paper "Inverting Generative Adversarial Renderer for Face Reconstruction"

StyleGAR TODO: add arxiv link Implementation of Inverting Generative Adversarial Renderer for Face Reconstruction TODO: for test Currently, some model

155 Oct 27, 2022
AI drive app that can help user become beautiful.

爱美丽 Beauty 简体中文 Features Beauty is an AI drive app that can help user become beautiful. it contain those functions: face score cheek face beauty repor

Starved Midnight 1 Jan 30, 2022
Repository for the paper "PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation", CVPR 2021.

PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation Code repository for the paper: PoseAug: A Differentiable Pose Augme

Pyjcsx 328 Dec 17, 2022
Official Implementation for the paper DeepFace-EMD: Re-ranking Using Patch-wise Earth Mover’s Distance Improves Out-Of-Distribution Face Identification

DeepFace-EMD: Re-ranking Using Patch-wise Earth Mover’s Distance Improves Out-Of-Distribution Face Identification Official Implementation for the pape

Anh M. Nguyen 36 Dec 28, 2022
CLIP2Video: Mastering Video-Text Retrieval via Image CLIP

CLIP2Video: Mastering Video-Text Retrieval via Image CLIP The implementation of paper CLIP2Video: Mastering Video-Text Retrieval via Image CLIP. CLIP2

168 Dec 29, 2022
DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification

DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification Created by Yongming Rao, Wenliang Zhao, Benlin Liu, Jiwen Lu, Jie Zhou, Ch

Yongming Rao 414 Jan 01, 2023
A Pytorch implementation of "Splitter: Learning Node Representations that Capture Multiple Social Contexts" (WWW 2019).

Splitter ⠀⠀ A PyTorch implementation of Splitter: Learning Node Representations that Capture Multiple Social Contexts (WWW 2019). Abstract Recent inte

Benedek Rozemberczki 201 Nov 09, 2022
Official re-implementation of the Calibrated Adversarial Refinement model described in the paper Calibrated Adversarial Refinement for Stochastic Semantic Segmentation

Official re-implementation of the Calibrated Adversarial Refinement model described in the paper Calibrated Adversarial Refinement for Stochastic Semantic Segmentation

Elias Kassapis 31 Nov 22, 2022
Example scripts for the detection of lanes using the ultra fast lane detection model in Tensorflow Lite.

TFlite Ultra Fast Lane Detection Inference Example scripts for the detection of lanes using the ultra fast lane detection model in Tensorflow Lite. So

Ibai Gorordo 12 Aug 27, 2022
YOLOv5🚀 reproduction by Guo Quanhao using PaddlePaddle

YOLOv5-Paddle YOLOv5 🚀 reproduction by Guo Quanhao using PaddlePaddle 支持AutoBatch 支持AutoAnchor 支持GPU Memory 快速开始 使用AIStudio高性能环境快速构建YOLOv5训练(PaddlePa

QuanHao Guo 20 Nov 14, 2022