Source code for 2021 ICCV paper "In-the-Wild Single Camera 3D Reconstruction Through Moving Water Surfaces"

Last update: Dec 06, 2022

Overview

In-the-Wild Single Camera 3D Reconstruction
Through Moving Water Surfaces

This is the PyTorch implementation for 2021 ICCV paper "In-the-Wild Single Camera 3D Reconstruction Through Moving Water Surfaces"

Project Page | Paper | Supplemental Material

In-the-Wild Single Camera 3D Reconstruction Through Moving Water Surfaces
Jinhui Xiong, Wolfgang Heidrich
KAUST
ICCV 2021 (Oral)

We propose a differentiable framework to estimate underwater scene geometry along with the time-varying water surface. The inputs to our model are a video sequence captured by a fixed camera. Dense correspondence from each frame to a world reference frame (selected from the input sequences) is pre-computed, ensuring the reconstruction is performed in a unified coordinate system. We feed the flow fields, together with initialized water surfaces and scene geometry (all are initialized as planar surfaces), into the framework, which incorporates ray casting, Snell’s law and multi-view triangulation. The gradients of the specially designed losses with respect to water surfaces and scene geometry are back-propagated, and all parameters are simultaneously optimized. The final result is a quality reconstruction of the underwater scene, along with an estimate of the time-varying water-air interface. The data shown here was captured in a public fountain environment.

Prerequisite

The code was tested with python>=3.7 & PyTorch>=1.3 & cuda>=10.0 on Nvidia RTX 2080 Ti
Minor change on the code if there is compatibility issue. It needs around 10 GB GPU memory.

Setup

conda create -n moving_water python=3.7
conda activate moving_water

conda install pytorch torchvision -c pytorch
conda install -c conda-forge opencv scikit-image
conda install -c anaconda scipy

Run the code

Please go to example folder, download the cached coefficient matrices (there are three matrices for each example) and execute:

python3 run.py

Citation

@inproceedings{xiong2021inthewild,
  title={In-the-Wild Single Camera 3D Reconstruction Through Moving Water Surfaces},
  author={Jinhui Xiong and Wolfgang Heidrich},
  year={2021},
  booktitle={ICCV}
}

Contact

Please contact Jinhui Xiong [email protected] if you have any question or comment.

Source code for 2021 ICCV paper "In-the-Wild Single Camera 3D Reconstruction Through Moving Water Surfaces"

Related tags

Overview

In-the-Wild Single Camera 3D Reconstruction
Through Moving Water Surfaces

Project Page | Paper | Supplemental Material

Prerequisite

Setup

Run the code

Citation

Contact

Owner

Official code of "Mitigating the Mutual Error Amplification for Semi-Supervised Object Detection"

Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attention Does Not Need O(n²) Memory"

Meli Data Challenge 2021 - First Place Solution

Apollo optimizer in tensorflow

Implementation of ICCV 2021 oral paper -- A Novel Self-Supervised Learning for Gaussian Mixture Model

PyTorch implementation of Towards Accurate Alignment in Real-time 3D Hand-Mesh Reconstruction (ICCV 2021).

Mmdetection3d Noted - MMDetection3D is an open source object detection toolbox based on PyTorch

Source Code for our paper: Understand me, if you refer to Aspect Knowledge: Knowledge-aware Gated Recurrent Memory Network

BEGAN in PyTorch

It is a system used to detect bone fractures. using techniques deep learning and image processing

DiSECt: Differentiable Simulator for Robotic Cutting

existing and custom freqtrade strategies supporting the new hyperstrategy format.

Repositório da disciplina de APC, no segundo semestre de 2021

Semantic similarity computation with different state-of-the-art metrics

Official implementation of the PICASO: Permutation-Invariant Cascaded Attentional Set Operator

Py-FEAT: Python Facial Expression Analysis Toolbox

A series of convenience functions to make basic image processing operations such as translation, rotation, resizing, skeletonization, and displaying Matplotlib images easier with OpenCV and Python.

Repository for Traffic Accident Benchmark for Causality Recognition (ECCV 2020)

Pytorch code for paper "Image Compressed Sensing Using Non-local Neural Network" TMM 2021.

MusicYOLO framework uses the object detection model, YOLOx, to locate notes in the spectrogram.