Aerial Depth Completion

This work is described in the letter "Aerial Single-View Depth Completion with Image-Guided Uncertainty Estimation", by Lucas Teixeira, Martin R. Oswald, Marc Pollefeys, Margarita Chli, published in the IEEE Robotics and Automation Letters (RA-L / ICRA) ETHZ Library link.

Video:

Presentation:

Citations:

If you use this Code or Aerial Dataset, please cite the following publication:

@article{Teixeira:etal:RAL2020,
    title   = {{Aerial Single-View Depth Completion with Image-Guided Uncertainty Estimation}},
    author  = {Lucas Teixeira and Martin R. Oswald and Marc Pollefeys and Margarita Chli},
    journal = {{IEEE} Robotics and Automation Letters ({RA-L})},
    doi     = {10.1109/LRA.2020.2967296},
    year    = {2020}
}

NYUv2, CAB and PVS datasets require further citation from their authors. During our research, we reformat and created ground-truth depth for the CAB and PVS datasets. This code also contains thirt-party networks used for comparison. Please also cite their authors properly in case of use.

Acknowledgment:

The authors thank Fangchang Ma and Abdelrahman Eldesokey for sharing their code that is partially used here. The authors also thanks the owner of the 3D models used to build the dataset. They are identified in each 3D model file.

Data and Simulator

Trained Models

Several trained models are available - here.

Datasets

• Aerial Dataset - link
• NYUv2 Dataset - link (preprocessed by Fangchang Ma and originally from Silberman et al. ECCV12)
• CAB Dataset - link (In this work, we created the depth information for the dataset originally published in Teixeira and Chli IROS16)
• PVS Dataset - link (In this work, we created the depth information for the dataset originally published in Restrepo et al. P&RS14)

To be used together by our code, the datasets need to be merged, this means that the content of the train folder of each dataset need to be place in a single train folder. The same happens with the eval folder.

Simulator

The Aerial Dataset was created using this simulator link.

3D Models

Most of the 3D models used to create the dataset can be download here. In the license files contain the authors of the 3D models. Some models were extended with a satellite image from Google Earth.

Running the code

Prerequisites

• PyTorch 1.0.1
• Python 3.6
• Plus dependencies

Testing Example

python3 main.py --evaluate "/media/lucas/lucas-ds2-1tb/tmp/model_best.pth.tar" --data-path "/media/lucas/lucas-ds2-1tb/dataset_big_v12"

Training Example

python3 main.py --data-path "/media/lucas/lucas-ds2-1tb/dataset_big_v12" --workers 8 -lr 0.00001 --batch-size 1 --dcnet-arch gudepthcompnet18 --training-mode dc1_only --criterion l2

python3 main.py --data-path "/media/lucas/lucas-ds2-1tb/dataset_big_v12" --workers 8 --criterion l2 --training-mode dc0-cf1-ln1 --dcnet-arch ged_depthcompnet --dcnet-pretrained /media/lucas/lucas-ds2-1tb/tmp/model_best.pth.tar:dc_weights --confnet-arch cbr3-c1 --confnet-pretrained /media/lucas/lucas-ds2-1tb/tmp/model_best.pth.tar:conf_weights --lossnet-arch ged_depthcompnet --lossnet-pretrained /media/lucas/lucas-ds2-1tb/tmp/model_best.pth.tar:lossdc_weights

Parameters

Contact

In case of any issue, fell free to contact me via email lteixeira at mavt.ethz.ch.