Toward Practical Monocular Indoor Depth Estimation

Cho-Ying Wu, Jialiang Wang, Michael Hall, Ulrich Neumann, Shuochen Su

[arXiv] [project site]

DistDepth

Our DistDepth is a highly robust monocular depth estimation approach for generic indoor scenes.

• Trained with stereo sequences without their groundtruth depth
• Structured and metric-accurate
• Run in an interactive rate with Laptop GPU
• Sim-to-real: trained on simulation and becomes transferrable to real scenes

Single Image Inference Demo

We test on Ubuntu 20.04 LTS with an laptop NVIDIA 2080 GPU (only GPU mode is supported).

Install packages

1. Use conda

   conda create --name distdepth python=3.8 conda activate distdepth

2. Install pre-requisite common packages. Go to https://pytorch.org/get-started/locally/ and install pytorch that is compatible to your computer. We test on pytorch v1.9.0 and cudatoolkit-11.1. (The codes should work under other v1.0+ versions)

   conda install pytorch==1.9.0 torchvision==0.10.0 torchaudio==0.9.0 cudatoolkit=11.3 -c pytorch -c conda-forge

3. Install other dependencies: opencv-python and matplotlib.

   pip install opencv-python, matplotlib

Download pretrained models

4. Download pretrained models [here] (ResNet152, 246MB).

5. Move the downloaded item under this folder, and then unzip it. You should be able to see a new folder 'ckpts' that contains the pretrained models.

6. Run

   python demo.py

7. Results will be stored under results/

Data

Download SimSIN [here]. For UniSIN and VA, please download at the [project site].

Depth-aware AR effects

Virtual object insertion:

Dragging objects along a trajectory:

Citation

@inproceedings{wu2022toward,
title={Toward Practical Monocular Indoor Depth Estimation},
author={Wu, Cho-Ying and Wang, Jialiang and Hall, Michael and Neumann, Ulrich and Su, Shuochen},
booktitle={CVPR},
year={2022}
}

License

DistDepth is CC-BY-NC licensed, as found in the LICENSE file.