Unofficial implementation of One-Shot Free-View Neural Talking Head Synthesis

Last update: Dec 30, 2022

Related tags

Overview

face-vid2vid

Usage

Dataset Preparation

cd datasets
wget https://yt-dl.org/downloads/latest/youtube-dl -O youtube-dl
chmod a+rx youtube-dl
python load_videos.py --workers=8
cd ..

Pretrained Headpose Estimator

300W-LP, alpha 1, robust to image quality

Put hopenet_robust_alpha1.pkl here

Train

python train.py --batch_size=4 --gpu_ids=0,1,2,3 --num_epochs=100 (--ckp=10)

On 2080Ti, setting batch_size=4 makes up gpu memory

Evaluate

Reconstruction：

python evaluate.py --ckp=99 --source=r --driving=datasets/vox/test/id10280#NXjT3732Ekg#001093#001192.mp4

The first frame is used as source by default

Motion transfer：

python evaluate.py --ckp=99 --source=test.png --driving=datasets/vox/test/id10280#NXjT3732Ekg#001093#001192.mp4

Example after training for 7 days on 4 2080Ti:

Face Frontalization：

python evaluate.py --ckp=99 --source=f --driving=datasets/vox/train/id10192#S5yV10aCP7A#003200#003334.mp4

Acknowlegement

Thanks to NV, Imaginaire, AliaksandrSiarohin and DeepHeadPose

Unofficial implementation of One-Shot Free-View Neural Talking Head Synthesis

Related tags

Overview

face-vid2vid

Usage

Dataset Preparation

Pretrained Headpose Estimator

Train

Evaluate

Acknowlegement

Owner

worstcoder

Code for NeurIPS 2021 paper: Invariant Causal Imitation Learning for Generalizable Policies

The source code of CVPR17 'Generative Face Completion'.

Code artifacts for the submission "Mind the Gap! A Study on the Transferability of Virtual vs Physical-world Testing of Autonomous Driving Systems"

Blender add-on: Add to Cameras menu: View → Camera, View → Add Camera, Camera → View, Previous Camera, Next Camera

Probabilistic Cross-Modal Embedding (PCME) CVPR 2021

Repository for the AugmentedPCA Python package.

Analysis of Antarctica sequencing samples contaminated with SARS-CoV-2

Source Code for our paper: Understand me, if you refer to Aspect Knowledge: Knowledge-aware Gated Recurrent Memory Network

Official code for 'Robust Siamese Object Tracking for Unmanned Aerial Manipulator' and offical introduction to UAMT100 benchmark

Code and data for ACL2021 paper Cross-Lingual Abstractive Summarization with Limited Parallel Resources.

Applications using the GTN library and code to reproduce experiments in "Differentiable Weighted Finite-State Transducers"

TensorFlow implementation of "TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?"

Mmrotate - OpenMMLab Rotated Object Detection Benchmark

FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks

Face Recognition & AI Based Smart Attendance Monitoring System.

Official code for the paper "Why Do Self-Supervised Models Transfer? Investigating the Impact of Invariance on Downstream Tasks".

Dynamic Head: Unifying Object Detection Heads with Attentions

Run Effective Large Batch Contrastive Learning on Limited Memory GPU

[ECCVW2020] Robust Long-Term Object Tracking via Improved Discriminative Model Prediction (RLT-DiMP)

Multivariate Time Series Forecasting with efficient Transformers. Code for the paper "Long-Range Transformers for Dynamic Spatiotemporal Forecasting."