Unofficial implementation of One-Shot Free-View Neural Talking Head Synthesis

Last update: Dec 30, 2022

Related tags

Overview

face-vid2vid

Usage

Dataset Preparation

cd datasets
wget https://yt-dl.org/downloads/latest/youtube-dl -O youtube-dl
chmod a+rx youtube-dl
python load_videos.py --workers=8
cd ..

Pretrained Headpose Estimator

300W-LP, alpha 1, robust to image quality

Put hopenet_robust_alpha1.pkl here

Train

python train.py --batch_size=4 --gpu_ids=0,1,2,3 --num_epochs=100 (--ckp=10)

On 2080Ti, setting batch_size=4 makes up gpu memory

Evaluate

Reconstruction：

python evaluate.py --ckp=99 --source=r --driving=datasets/vox/test/id10280#NXjT3732Ekg#001093#001192.mp4

The first frame is used as source by default

Motion transfer：

python evaluate.py --ckp=99 --source=test.png --driving=datasets/vox/test/id10280#NXjT3732Ekg#001093#001192.mp4

Example after training for 7 days on 4 2080Ti:

Face Frontalization：

python evaluate.py --ckp=99 --source=f --driving=datasets/vox/train/id10192#S5yV10aCP7A#003200#003334.mp4

Acknowlegement

Thanks to NV, Imaginaire, AliaksandrSiarohin and DeepHeadPose

Unofficial implementation of One-Shot Free-View Neural Talking Head Synthesis

Related tags

Overview

face-vid2vid

Usage

Dataset Preparation

Pretrained Headpose Estimator

Train

Evaluate

Acknowlegement

Owner

worstcoder

Customised to detect objects automatically by a given model file(onnx)

ZEBRA: Zero Evidence Biometric Recognition Assessment

Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.

Sparse R-CNN: End-to-End Object Detection with Learnable Proposals, CVPR2021

Official PyTorch code of DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization (ICCV 2021 Oral).

Atomistic Line Graph Neural Network

Python code for the paper How to scale hyperparameters for quickshift image segmentation

Madanalysis5 - A package for event file analysis and recasting of LHC results

Code for ACM MM 2020 paper "NOH-NMS: Improving Pedestrian Detection by Nearby Objects Hallucination"

This is the official PyTorch implementation of the CVPR 2020 paper "TransMoMo: Invariance-Driven Unsupervised Video Motion Retargeting".

Space Time Recurrent Memory Network - Pytorch

Deep Semisupervised Multiview Learning With Increasing Views (IEEE TCYB 2021, PyTorch Code)

Code for ECCV 2020 paper "Contacts and Human Dynamics from Monocular Video".

Starter Code for VALUE benchmark

In this project we use both Resnet and Self-attention layer for cat, dog and flower classification.

A unified 3D Transformer Pipeline for visual synthesis

Machine Learning Framework for Operating Systems - Brings ML to Linux kernel

Catalyst.Detection

Notepy is a full-featured Notepad Python app

Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed model training.