Official PyTorch Implementation of paper "NeLF: Neural Light-transport Field for Single Portrait View Synthesis and Relighting", EGSR 2021.

Related tags

Text Data & NLPnelf
Overview

NeLF: Neural Light-transport Field for Single Portrait View Synthesis and Relighting

Official PyTorch Implementation of paper "NeLF: Neural Light-transport Field for Single Portrait View Synthesis and Relighting", EGSR 2021.

Tiancheng Sun1*, Kai-En Lin1*, Sai Bi2, Zexiang Xu2, Ravi Ramamoorthi1

1University of California, San Diego, 2Adobe Research

*Equal contribution

Project Page | Paper | Pretrained models | Validation data | Rendering script

Requirements

Install required packages

Make sure you have up-to-date NVIDIA drivers supporting CUDA 11.1 (10.2 could work but need to change cudatoolkit package accordingly)

Run

conda env create -f environment.yml
conda activate pixelnerf

The following packages are used:

  • PyTorch (1.7 & 1.9.0 Tested)

  • OpenCV-Python

  • matplotlib

  • numpy

  • tqdm

OS system: Ubuntu 20.04

Download CelebAMask-HQ dataset link

  1. Download the dataset

  2. Remove background with the provided masks in the dataset

  3. Downsample the dataset to 512x512

  4. Store the resulting data in [path_to_data_directory]/CelebAMask

    Following this data structure

    [path_to_data_directory] --- data --- CelebAMask --- 0.jpg
                                       |              |- 1.jpg
                                       |              |- 2.jpg
                                       |              ...
                                       |- blender_both --- sub001
                                       |                |- sub002
                                       |                ...
    
    

(Optional) Download and render FaceScape dataset link

Due to FaceScape's license, we cannot release the full dataset. Instead, we will release our rendering script.

  1. Download the dataset

  2. Install Blender link

  3. Run rendering script link

Usage

Testing

  1. Download our pretrained checkpoint and testing data. Extract the content to [path_to_data_directory]. The data structure should look like this:

    [path_to_data_directory] --- data --- CelebAMask
                              |        |- blender_both
                              |        |- blender_view
                              |        ...
                              |- data_results --- nelf_ft
                              |- data_test --- validate_0
                                            |- validate_1
                                            |- validate_2
    
  2. In arg/__init__.py, setup data path by changing base_path

  3. Run python run_test.py nelf_ft [validation_data_name] [#iteration_for_the_model]

    e.g. python run_test.py nelf_ft validate_0 500000

  4. The results are stored in [path_to_data_directory]/data_test/[validation_data_name]/results

Training

Due to FaceScape's license, we are not allowed to release the full dataset. We will use validation data to run the following example.

  1. Download our validation data. Extract the content to [path_to_data_directory]. The data structure should look like this:

    [path_to_data_directory] --- data --- CelebAMask
                              |        |- blender_both
                              |        |- blender_view
                              |        ...
                              |- data_results --- nelf_ft
                              |- data_test --- validate_0
                                            |- validate_1
                                            |- validate_2
    

    (Optional) Run rendering script and render your own data.

    Remember to change line 35~42 and line 45, 46 in arg/config_nelf_ft.py accordingly.

  2. In arg/__init__.py, setup data path by changing base_path

  3. Run python run_train.py nelf_ft

  4. The intermediate results and model checkpoints are saved in [path_to_data_directory]/data_results/nelf_ft

Configs

The following config files can be found inside arg folder

Citation

@inproceedings {sun2021nelf,
    booktitle = {Eurographics Symposium on Rendering},
    title = {NeLF: Neural Light-transport Field for Portrait View Synthesis and Relighting},
    author = {Sun, Tiancheng and Lin, Kai-En and Bi, Sai and Xu, Zexiang and Ramamoorthi, Ravi},
    year = {2021},
}
Owner
Ken Lin
Ken Lin
Simple, Pythonic, text processing--Sentiment analysis, part-of-speech tagging, noun phrase extraction, translation, and more.

TextBlob: Simplified Text Processing Homepage: https://textblob.readthedocs.io/ TextBlob is a Python (2 and 3) library for processing textual data. It

Steven Loria 8.4k Dec 26, 2022
LegalNLP - Natural Language Processing Methods for the Brazilian Legal Language

LegalNLP - Natural Language Processing Methods for the Brazilian Legal Language ⚖️ The library of Natural Language Processing for Brazilian legal lang

Felipe Maia Polo 125 Dec 20, 2022
Write Alphabet, Words and Sentences with your eyes.

The-Next-Gen-AI-Eye-Writer The Eye tracking Technique has become one of the most popular techniques within the human and computer interaction era, thi

Rohan Kasabe 2 Apr 05, 2022
Pretrained Japanese BERT models

Pretrained Japanese BERT models This is a repository of pretrained Japanese BERT models. The models are available in Transformers by Hugging Face. Mod

Inui Laboratory 387 Dec 30, 2022
A framework for implementing federated learning

This is partly the reproduction of the paper of [Privacy-Preserving Federated Learning in Fog Computing](DOI: 10.1109/JIOT.2020.2987958. 2020)

DavidChen 46 Sep 23, 2022
In this project, we compared Spanish BERT and Multilingual BERT in the Sentiment Analysis task.

Applying BERT Fine Tuning to Sentiment Classification on Amazon Reviews Abstract Sentiment analysis has made great progress in recent years, due to th

Alexander Leonardo Lique Lamas 5 Jan 03, 2022
Just a basic Telegram AI chat bot written in Python using Pyrogram.

Nikko ChatBot Just a basic Telegram AI chat bot written in Python using Pyrogram. Requirements Python 3.7 or higher. A bot token. Installation $ https

ʀᴇxɪɴᴀᴢᴏʀ 2 Oct 21, 2022
Search msDS-AllowedToActOnBehalfOfOtherIdentity

前言 现在进行RBCD的攻击手段主要是搜索mS-DS-CreatorSID,如果机器的创建者是我们可控的话,那就可以修改对应机器的msDS-AllowedToActOnBehalfOfOtherIdentity,利用工具SharpAllowedToAct-Modify 那我们索性也试试搜索所有计算机

Jumbo 26 Dec 05, 2022
profile tools for pytorch nn models

nnprof Introduction nnprof is a profile tool for pytorch neural networks. Features multi profile mode: nnprof support 4 profile mode: Layer level, Ope

Feng Wang 42 Jul 09, 2022
Generate product descriptions, blogs, ads and more using GPT architecture with a single request to TextCortex API a.k.a Hemingwai

TextCortex - HemingwAI Generate product descriptions, blogs, ads and more using GPT architecture with a single request to TextCortex API a.k.a Hemingw

TextCortex AI 27 Nov 28, 2022
Binary LSTM model for text classification

Text Classification The purpose of this repository is to create a neural network model of NLP with deep learning for binary classification of texts re

Nikita Elenberger 1 Mar 11, 2022
Repository for Graph2Pix: A Graph-Based Image to Image Translation Framework

Graph2Pix: A Graph-Based Image to Image Translation Framework Installation Install the dependencies in env.yml $ conda env create -f env.yml $ conda a

18 Nov 17, 2022
Twitter Sentiment Analysis using #tag, words and username

Twitter Sentment Analysis Web App using #tag, words and username to fetch data finds Insides of data and Tells Sentiment of the perticular #tag, words or username.

Kumar Saksham 26 Dec 25, 2022
Implementation of the Hybrid Perception Block and Dual-Pruned Self-Attention block from the ITTR paper for Image to Image Translation using Transformers

ITTR - Pytorch Implementation of the Hybrid Perception Block (HPB) and Dual-Pruned Self-Attention (DPSA) block from the ITTR paper for Image to Image

Phil Wang 17 Dec 23, 2022
This repository details the steps in creating a Part of Speech tagger using Trigram Hidden Markov Models and the Viterbi Algorithm without using external libraries.

POS-Tagger This repository details the creation of a Part-of-Speech tagger using Trigram Hidden Markov Models to predict word tags in a word sequence.

Raihan Ahmed 1 Dec 09, 2021
Paddle2.x version AI-Writer

Paddle2.x 版本AI-Writer 用魔改 GPT 生成网文。Tuned GPT for novel generation.

yujun 74 Jan 04, 2023
Simple multilingual lemmatizer for Python, especially useful for speed and efficiency

Simplemma: a simple multilingual lemmatizer for Python Purpose Lemmatization is the process of grouping together the inflected forms of a word so they

Adrien Barbaresi 70 Dec 29, 2022
Modeling cumulative cases of Covid-19 in the US during the Covid 19 Delta wave using Bayesian methods.

Introduction The goal of this analysis is to find a model that fits the observed cumulative cases of COVID-19 in the US, starting in Mid-July 2021 and

Alexander Keeney 1 Jan 05, 2022
jel - Japanese Entity Linker - is Bi-encoder based entity linker for japanese.

jel: Japanese Entity Linker jel - Japanese Entity Linker - is Bi-encoder based entity linker for japanese. Usage Currently, link and question methods

izuna385 10 Jan 06, 2023
A telegram bot to translate 100+ Languages

🔥 GOOGLE TRANSLATER 🔥 The owner would not be responsible for any kind of bans due to the bot. • ⚡ INSTALLING ⚡ • • 🔰 Deploy To Railway 🔰 • • ✅ OFF

Aɴᴋɪᴛ Kᴜᴍᴀʀ 5 Dec 20, 2021