Skip to content

wonjongg/StyleCariGAN

Repository files navigation

StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation

License CC BY_NC

teaser

📝 This repository contains the official PyTorch implementation of the following paper:

StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation
Wonjong Jang, Gwangjin Ju, Yucheol Jung, Jiaolong Yang, Xin Tong, Seungyong Lee, SIGGRAPH 2021

🚀 >> Project page
🚀 >> Fast-forward video

Overview

method

Explanation
The key component of our method is shape exaggeration blocks that are used for modulating coarse layer feature maps of StyleGAN to produce desirable caricature shape exaggerations. We first build a layer-mixed StyleGAN for photo-to-caricature style conversion by swapping fine layers of the StyleGAN for photos to the corresponding layers of the StyleGAN trained to generate caricatures. Given an input photo, the layer-mixed model produces detailed color stylization for a caricature but without shape exaggerations. We then append shape exaggeration blocks to the coarse layers of the layer-mixed model and train the blocks to create shape exaggerations while preserving the characteristic appearances of the input.

Requirements

✔️ PyTorch 1.3.1
✔️ torchvision 0.4.2
✔️ CUDA 10.1/10.2
✔️ dlib 19.22.0
✔️ requests 2.23.0
✔️ tqdm 4.46.2

⚠️ If you are using Anaconda environment and get errors regarding compiler version mismatch, check issue #1.

Usage

First download pre-trained model weights:

bash ./download.sh

Train

python -m torch.distributed.launch --nproc_per_node=N_GPU train.py --name EXPERIMENT_NAME --freeze_D

Test

Test on user's input images:

python test.py --ckpt CHECKPOINT_PATH --input_dir INPUT_IMAGE_PATH --output_dir OUTPUT_CARICATURE_PATH --invert_images

We provide some sample images. Test on sample images:

python test.py --ckpt CHECKPOINT_PATH --input_dir examples/samples --output_dir examples/results --invert_images

It inverts latent codes from input photos and generates caricatures from latent codes.

Examples

Input image Output caricature
img1 cari1
img2 cari2
img3 cari3
img4 cari4

Citation

If you find this code useful, please consider citing:

@article{Jang2021StyleCari,
  author    = {Wonjong Jang and Gwangjin Ju and Yucheol Jung and Jiaolong Yang and Xin Tong and Seungyong Lee},
  title     = {StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation},
  booktitle = {ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH)},
  publisher = {ACM},
  volume = {40},
  number = {4},
  year = {2021}
}

Download pre-trained models

🏷️ StyleCariGAN
🏷️ Photo-StyleGAN (generator_ffhq.pt)
🏷️ Caricature-StyleGAN (generator_cari.pt)
🏷️ Photo-Attribute-Classifier (photo_resnet.pth)
🏷️ Cari-Attribute-Classifier (cari_resnet.pth)

Contact

📫 You can have contact with wonjong@postech.ac.kr or ycjung@postech.ac.kr

License

This software is being made available under the terms in the LICENSE file.

Any exemptions to these terms require a license from the Pohang University of Science and Technology.

Credits

❤️ Our code is based on the official StyleGAN2 implementation and rosinality's StyleGAN2-pytorch code
❤️ Specially thanks to CJWBW who ported our project to Replicate.

About

[SIGGRAPH 2021] Official implementation of "StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation"

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published