Unofficial implementation of One-Shot Free-View Neural Talking Head Synthesis

Last update: Dec 30, 2022

Related tags

Overview

face-vid2vid

Usage

Dataset Preparation

cd datasets
wget https://yt-dl.org/downloads/latest/youtube-dl -O youtube-dl
chmod a+rx youtube-dl
python load_videos.py --workers=8
cd ..

Pretrained Headpose Estimator

300W-LP, alpha 1, robust to image quality

Put hopenet_robust_alpha1.pkl here

Train

python train.py --batch_size=4 --gpu_ids=0,1,2,3 --num_epochs=100 (--ckp=10)

On 2080Ti, setting batch_size=4 makes up gpu memory

Evaluate

Reconstruction：

python evaluate.py --ckp=99 --source=r --driving=datasets/vox/test/id10280#NXjT3732Ekg#001093#001192.mp4

The first frame is used as source by default

Motion transfer：

python evaluate.py --ckp=99 --source=test.png --driving=datasets/vox/test/id10280#NXjT3732Ekg#001093#001192.mp4

Example after training for 7 days on 4 2080Ti:

Face Frontalization：

python evaluate.py --ckp=99 --source=f --driving=datasets/vox/train/id10192#S5yV10aCP7A#003200#003334.mp4

Acknowlegement

Thanks to NV, Imaginaire, AliaksandrSiarohin and DeepHeadPose

Unofficial implementation of One-Shot Free-View Neural Talking Head Synthesis

Related tags

Overview

face-vid2vid

Usage

Dataset Preparation

Pretrained Headpose Estimator

Train

Evaluate

Acknowlegement

Owner

worstcoder

In this project we investigate the performance of the SetCon model on realistic video footage. Therefore, we implemented the model in PyTorch and tested the model on two example videos.

The Body Part Regression (BPR) model translates the anatomy in a radiologic volume into a machine-interpretable form.

Code for NeurIPS 2021 paper: Invariant Causal Imitation Learning for Generalizable Policies

Transferable Unrestricted Attacks, which won 1st place in CVPR’21 Security AI Challenger: Unrestricted Adversarial Attacks on ImageNet.

AdaMML: Adaptive Multi-Modal Learning for Efficient Video Recognition

RepMLP: Re-parameterizing Convolutions into Fully-connected Layers for Image Recognition

RIFE - Real-Time Intermediate Flow Estimation for Video Frame Interpolation

[WWW 2021] Source code for "Graph Contrastive Learning with Adaptive Augmentation"

SwinTrack: A Simple and Strong Baseline for Transformer Tracking

Start-to-finish tutorial for interactive music co-creation in PyTorch and Tensorflow.js

Optimizers-visualized - Visualization of different optimizers on local minimas and saddle points.

Train an RL agent to execute natural language instructions in a 3D Environment (PyTorch)

Think Big, Teach Small: Do Language Models Distil Occam’s Razor?

This is the official PyTorch implementation of our paper: "Artistic Style Transfer with Internal-external Learning and Contrastive Learning".

Pytorch implementation of the paper "Topic Modeling Revisited: A Document Graph-based Neural Network Perspective"

Stacked Hourglass Network with a Multi-level Attention Mechanism: Where to Look for Intervertebral Disc Labeling

Code for ACM MM2021 paper "Complementary Trilateral Decoder for Fast and Accurate Salient Object Detection"

Official implementation of "Generating 3D Molecules for Target Protein Binding"

ilpyt: imitation learning library with modular, baseline implementations in Pytorch

Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition