Deep Illuminator is a data augmentation tool designed for image relighting. It can be used to easily and efficiently generate a wide range of illumination variants of a single image.

Overview

Deep Illuminator

Deep Illuminator is a data augmentation tool designed for image relighting. It can be used to easily and efficiently generate a wide range of illumination variants of a single image. It has been tested with several datasets and models and has been shown to succesfully improve performance. It has a built in visualizer created with Streamlit to preview how the target image can be relit. This tool has an accompanying paper.

Example Augmentations

Usage

The simplest method to use this tool is through Docker Hub:

docker pull kartvel/deep-illuminator

Visualizer

Once you have the Deep Illuminator image run the following command to launch the visualizer:

docker run -it --rm  --gpus all \
-p 8501:8501 --entrypoint streamlit \ 
kartvel/deep-illuminator run streamlit/streamlit_app.py

You will be able to interact with it on localhost:8501. Note: If you do not have NVIDIA gpu support enabled for docker simply remove the --gpus all option.

Generating Variants

It is possible to quickly generate multiple variants for images contained in a directory by using the following command:

docker run -it --rm --gpus all \                                                                                               ─╯
-v /path/to/input/images:/app/probe_relighting/originals \
-v /path/to/save/directory:/app/probe_relighting/output \
kartvel/deep-illuminator --[options]

Options

Option Values Description
mode ['synthetic', 'mid'] Selecting the style of probes used as a relighting guide.
step int Increment for the granularity of relighted images. max mid: 24, max synthetic: 360

Buidling Docker image or running without a container

Please read the following for other options: instructions

Benchmarks

Improved performance of R2D2 for [email protected] on HPatches

Training Dataset Overall Viewpoint Illumination
COCO - Original 71.0 65.4 77.1
COCO - Augmented 72.2 (+1.7%) 65.7 (+0.4%) 79.2 (+2.7%)
VIDIT - Original 66.7 60.5 73.4
VIDIT - Augmented 69.2 (+3.8%) 60.9 (+0.6%) 78.1 (+6.4%)
Aachen - Original 69.4 64.1 75.0
Aachen - Augmented 72.6 (+4.6%) 66.1 (+3.1%) 79.6 (+6.1%)

Improved performance of R2D2 for the Long-Term Visual Localization challenge on Aachen v1.1

Training Dataset 0.25m, 2° 0.5m, 5° 5m, 10°
COCO - Original 62.3 77.0 79.5
COCO - Augmented 65.4 (+5.0%) 83.8 (+8.8%) 92.7 (+16%)
VIDIT - Original 40.8 53.4 61.3
VIDIT - Augmented 53.9 (+32%) 71.2 (+33%) 83.2(+36%)
Aachen - Original 60.7 72.8 83.8
Aachen - Augmented 63.4 (+4.4%) 81.7 (+12%) 92.1 (+9.9%)

Acknowledgment

The developpement of the VAE for the visualizer was made possible by the PyTorch-VAE repository.

Bibtex

If you use this code in your project, please consider citing the following paper:

@misc{chogovadze2021controllable,
      title={Controllable Data Augmentation Through Deep Relighting}, 
      author={George Chogovadze and Rémi Pautrat and Marc Pollefeys},
      year={2021},
      eprint={2110.13996},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}
This repo is a PyTorch implementation for Paper "Unsupervised Learning for Cuboid Shape Abstraction via Joint Segmentation from Point Clouds"

Unsupervised Learning for Cuboid Shape Abstraction via Joint Segmentation from Point Clouds This repository is a PyTorch implementation for paper: Uns

Kaizhi Yang 42 Dec 09, 2022
[NeurIPS 2021] Introspective Distillation for Robust Question Answering

Introspective Distillation (IntroD) This repository is the Pytorch implementation of our paper "Introspective Distillation for Robust Question Answeri

Yulei Niu 13 Jul 26, 2022
Machine Learning automation and tracking

The Open-Source MLOps Orchestration Framework MLRun is an open-source MLOps framework that offers an integrative approach to managing your machine-lea

873 Jan 04, 2023
Implementations of orthogonal and semi-orthogonal convolutions in the Fourier domain with applications to adversarial robustness

Orthogonalizing Convolutional Layers with the Cayley Transform This repository contains implementations and source code to reproduce experiments for t

CMU Locus Lab 36 Dec 30, 2022
Pgn2tex - Scripts to convert pgn files to latex document. Useful to build books or pdf from pgn studies

Pgn2Latex (WIP) A simple script to make pdf from pgn files and studies. It's sti

12 Jul 23, 2022
Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning using 🤗 transformers

hierarchical-transformer-1d Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning using 🤗 transformers In Progress!! 2021.

MyungHoon Jin 7 Nov 06, 2022
A list of multi-task learning papers and projects.

This page contains a list of papers on multi-task learning for computer vision. Please create a pull request if you wish to add anything. If you are interested, consider reading our recent survey pap

svandenh 297 Dec 17, 2022
Simple converter for deploying Stable-Baselines3 model to TFLite and/or Coral

Running SB3 developed agents on TFLite or Coral Introduction I've been using Stable-Baselines3 to train agents against some custom Gyms, some of which

Gary Briggs 16 Oct 11, 2022
Controlling Hill Climb Racing with Hand Tacking

Controlling Hill Climb Racing with Hand Tacking Opened Palm for Gas Closed Palm for Brake

Rohit Ingole 3 Jan 18, 2022
Exploring whether attention is necessary for vision transformers

Do You Even Need Attention? A Stack of Feed-Forward Layers Does Surprisingly Well on ImageNet Paper/Report TL;DR We replace the attention layer in a v

Luke Melas-Kyriazi 461 Jan 07, 2023
Human segmentation models, training/inference code, and trained weights, implemented in PyTorch

Human-Segmentation-PyTorch Human segmentation models, training/inference code, and trained weights, implemented in PyTorch. Supported networks UNet: b

Thuy Ng 474 Dec 19, 2022
Code implementation of Data Efficient Stagewise Knowledge Distillation paper.

Data Efficient Stagewise Knowledge Distillation Table of Contents Data Efficient Stagewise Knowledge Distillation Table of Contents Requirements Image

IvLabs 112 Dec 02, 2022
Pyramid Pooling Transformer for Scene Understanding

Pyramid Pooling Transformer for Scene Understanding Requirements: torch 1.6+ torchvision 0.7.0 timm==0.3.2 Validated on torch 1.6.0, torchvision 0.7.0

Yu-Huan Wu 119 Dec 29, 2022
basic tutorial on pytorch

Quick Tutorial on PyTorch PyTorch Basics Linear Regression Logistic Regression Artificial Neural Networks Convolutional Neural Networks Recurrent Neur

7 Sep 15, 2022
Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes (CVPR 2021 Oral)

Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Surfaces Official code release for NGLOD. For technical details, please refer t

659 Dec 27, 2022
One line to host them all. Bootstrap your image search case in minutes.

One line to host them all. Bootstrap your image search case in minutes. Survey NOW gives the world access to customized neural image search in just on

Jina AI 403 Dec 30, 2022
Malmo Collaborative AI Challenge - Team Pig Catcher

The Malmo Collaborative AI Challenge - Team Pig Catcher Approach The challenge involves 2 agents who can either cooperate or defect. The optimal polic

Kai Arulkumaran 66 Jun 29, 2022
Collect super-resolution related papers, data, repositories

Collect super-resolution related papers, data, repositories

WangChaofeng 1.7k Jan 03, 2023
A PyTorch implementation for Unsupervised Domain Adaptation by Backpropagation(DANN), support Office-31 and Office-Home dataset

DANN A PyTorch implementation for Unsupervised Domain Adaptation by Backpropagation Prerequisites Linux or OSX NVIDIA GPU + CUDA (may CuDNN) and corre

8 Apr 16, 2022
An open-source Deep Learning Engine for Healthcare that aims to treat & prevent major diseases

AlphaCare Background AlphaCare is a work-in-progress, open-source Deep Learning Engine for Healthcare that aims to treat and prevent major diseases. T

Siraj Raval 44 Nov 05, 2022