Official implementation of NLOS-OT: Passive Non-Line-of-Sight Imaging Using Optimal Transport (IEEE TIP, accepted)

Related tags

Deep LearningNLOS-OT
Overview

NLOS-OT

Official implementation of NLOS-OT: Passive Non-Line-of-Sight Imaging Using Optimal Transport (IEEE TIP, accepted)

Description

In this repository, we release the NLOS-OT code in Pytorch as well as the passive NLOS imaging dataset NLOS-Passive.

  • Problem statement: Passive NLOS imaging

  • NLOS-OT architecture

  • The reconstruction results of NLOS-OT trained by specific dataset without partial occluder

  • The generalization results of NLOS-OT trained by dataset only from STL-10 with unknown partial occluder

Installation

  1. install required packages

  2. clone the repo

Prepare Data

  1. Download dataset

You can download each group in NLOS-Passive through the link below. Please note that a compressed package (.zip or .z01+.zip) represents a group of measured data.

link:https://pan.baidu.com/s/19Q48BWm1aJQhIt6BF9z-uQ

code:j3p2

If the link fails, please feel free to contact me.

  1. Organize the files structure of the dataset

Demo / Evaluate

Before that, you should have installed the required packages and organized the data set according to the appropriate file structure.

  1. Download pretrained pth

  2. run the test.py

Train

Before that, you should have installed the required packages and organized the data set according to the appropriate file structure.

Citation

If you find our work and code helpful, please consider cite:

We thank the following great works:

  • DeblurGAN, pix2pix: Our code is based on the framework provided by the two repos.

  • IntroVAE: The encoder and decoder in NLOS-OT are based on IntroVAE.

  • AE-OT, AEOT-GAN: The idea of using OT to complete passive NLOS imaging tasks in NLOS-OT comes from the two works.

If you find them helpful, please cite:

@inproceedings{kupynDeblurGANBlindMotion2018,
	title = {{DeblurGAN}: {Blind} {Motion} {Deblurring} {Using} {Conditional} {Adversarial} {Networks}},
	booktitle = {2018 {IEEE} {Conference} on {Computer} {Vision} and {Pattern} {Recognition} ({CVPR})},
	author = {Kupyn, Orest and Budzan, Volodymyr and Mykhailych, Mykola and Mishkin, Dmytro and Matas, Jiri},
	year = {2018},
}

@inproceedings{isolaImagetoimageTranslationConditional2017,
	title = {Image-to-image translation with conditional adversarial networks},
	booktitle = {2017 {IEEE} {Conference} on {Computer} {Vision} and {Pattern} {Recognition} ({CVPR})},
	publisher = {IEEE},
	author = {Isola, Phillip and Zhu, Jun-Yan and Zhou, Tinghui and Efros, Alexei A.},
	year = {2017},
	pages = {5967--5976},
}

@inproceedings{huang_introvae_2018,
	title = {{IntroVAE}: {Introspective} {Variational} {Autoencoders} for {Photographic} {Image} {Synthesis}},
	shorttitle = {{IntroVAE}},
	urldate = {2020-07-14},
	booktitle = {Advances in neural information processing systems},
	author = {Huang, Huaibo and Li, Zhihang and He, Ran and Sun, Zhenan and Tan, Tieniu},
	month = oct,
	year = {2018}
}

@article{an_ae-ot-gan_2020,
	title = {{AE}-{OT}-{GAN}: {Training} {Gans} from {Data} {Specific} {Latent} {Distribution}},
	shorttitle = {Ae-{Ot}-{Gan}},
	journal = {arXiv},
	author = {An, Dongsheng and Guo, Yang and Zhang, Min and Qi, Xin and Lei, Na and Yau, Shing-Tung and Gu, Xianfeng},
	year = {2020}
}

@inproceedings{an_ae-ot_2020,
	title = {{AE}-{OT}: {A} {NEW} {GENERATIVE} {MODEL} {BASED} {ON} {EX}- {TENDED} {SEMI}-{DISCRETE} {OPTIMAL} {TRANSPORT}},
	language = {en},
	author = {An, Dongsheng and Guo, Yang and Lei, Na and Luo, Zhongxuan and Yau, Shing-Tung and Gu, Xianfeng},
	year = {2020},
	pages = {19},
}
Owner
Ruixu Geng(耿瑞旭)
Undergraduate 2015 - 2019 (Expected), Information and Communication Engineering, UESTC
Ruixu Geng(耿瑞旭)
Official Chainer implementation of GP-GAN: Towards Realistic High-Resolution Image Blending (ACMMM 2019, oral)

GP-GAN: Towards Realistic High-Resolution Image Blending (ACMMM 2019, oral) [Project] [Paper] [Demo] [Related Work: A2RL (for Auto Image Cropping)] [C

Wu Huikai 402 Dec 27, 2022
AITom is an open-source platform for AI driven cellular electron cryo-tomography analysis.

AITom Introduction AITom is an open-source platform for AI driven cellular electron cryo-tomography analysis. AITom is originated from the tomominer l

93 Jan 02, 2023
A PyTorch implementation of the paper "Semantic Image Synthesis via Adversarial Learning" in ICCV 2017

Semantic Image Synthesis via Adversarial Learning This is a PyTorch implementation of the paper Semantic Image Synthesis via Adversarial Learning. Req

Seonghyeon Nam 146 Nov 25, 2022
CVPR2021: Temporal Context Aggregation Network for Temporal Action Proposal Refinement

Temporal Context Aggregation Network - Pytorch This repo holds the pytorch-version codes of paper: "Temporal Context Aggregation Network for Temporal

Zhiwu Qing 63 Sep 27, 2022
SenseNet is a sensorimotor and touch simulator for deep reinforcement learning research

SenseNet is a sensorimotor and touch simulator for deep reinforcement learning research

59 Feb 25, 2022
Self-supervised Point Cloud Prediction Using 3D Spatio-temporal Convolutional Networks

Self-supervised Point Cloud Prediction Using 3D Spatio-temporal Convolutional Networks This is a Pytorch-Lightning implementation of the paper "Self-s

Photogrammetry & Robotics Bonn 111 Dec 06, 2022
Lexical Substitution Framework

LexSubGen Lexical Substitution Framework This repository contains the code to reproduce the results from the paper: Arefyev Nikolay, Sheludko Boris, P

Samsung 37 Sep 15, 2022
PyTorch implementation of DeepUME: Learning the Universal Manifold Embedding for Robust Point Cloud Registration (BMVC 2021)

DeepUME: Learning the Universal Manifold Embedding for Robust Point Cloud Registration [video] [paper] [supplementary] [data] [thesis] Introduction De

Natalie Lang 10 Dec 14, 2022
A gesture recognition system powered by OpenPose, k-nearest neighbours, and local outlier factor.

OpenHands OpenHands is a gesture recognition system powered by OpenPose, k-nearest neighbours, and local outlier factor. Currently the system can iden

Paul Treanor 12 Jan 10, 2022
This was initially the repo for the project of [email protected] of Asaf Mazar, Millad Kassaie and Georgios Chochlakis named "Powered by the Will? Exploring Lay Theories of Behavior Change through Social Media"

Subreddit Analysis This repo includes tools for Subreddit analysis, originally developed for our class project of PSYC 626 in USC, titled "Powered by

Georgios Chochlakis 1 Dec 17, 2021
The modify PyTorch version of Siam-trackers which are speed-up by TensorRT.

SiamTracker-with-TensorRT The modify PyTorch version of Siam-trackers which are speed-up by TensorRT or ONNX. [Updating...] Examples demonstrating how

9 Dec 13, 2022
Code for generating a single image pretraining dataset

Single Image Pretraining of Visual Representations As shown in the paper A critical analysis of self-supervision, or what we can learn from a single i

Yuki M. Asano 12 Dec 19, 2022
Code of U2Fusion: a unified unsupervised image fusion network for multiple image fusion tasks, including multi-modal, multi-exposure and multi-focus image fusion.

U2Fusion Code of U2Fusion: a unified unsupervised image fusion network for multiple image fusion tasks, including multi-modal (VIS-IR, medical), multi

Han Xu 129 Dec 11, 2022
Consistency Regularization for Adversarial Robustness

Consistency Regularization for Adversarial Robustness Official PyTorch implementation of Consistency Regularization for Adversarial Robustness by Jiho

40 Dec 17, 2022
MAVE: : A Product Dataset for Multi-source Attribute Value Extraction

MAVE: : A Product Dataset for Multi-source Attribute Value Extraction The dataset contains 3 million attribute-value annotations across 1257 unique ca

Google Research Datasets 89 Jan 08, 2023
Stacs-ci - A set of modules to enable integration of STACS with commonly used CI / CD systems

Static Token And Credential Scanner CI Integrations What is it? STACS is a YARA

STACS 18 Aug 04, 2022
Feup-csr - Repository holding my group's submission to the CSR project competition

CSR Competições de Swarm Robotics Swarm Robotics Competitions This repository holds the files submitted for the CSR project competition. Project group

Nuno Pereira 1 Jan 04, 2022
Codebase for Image Classification Research, written in PyTorch.

pycls pycls is an image classification codebase, written in PyTorch. It was originally developed for the On Network Design Spaces for Visual Recogniti

Facebook Research 2k Jan 01, 2023
TalkNet 2: Non-Autoregressive Depth-Wise Separable Convolutional Model for Speech Synthesis with Explicit Pitch and Duration Prediction.

TalkNet 2 [WIP] TalkNet 2: Non-Autoregressive Depth-Wise Separable Convolutional Model for Speech Synthesis with Explicit Pitch and Duration Predictio

Rishikesh (ऋषिकेश) 69 Dec 17, 2022
A data-driven approach to quantify the value of classifiers in a machine learning ensemble.

Documentation | External Resources | Research Paper Shapley is a Python library for evaluating binary classifiers in a machine learning ensemble. The

Benedek Rozemberczki 188 Dec 29, 2022