Office source code of paper UniFuse: Unidirectional Fusion for 360$^\circ$ Panorama Depth Estimation

Last update: Dec 26, 2022

Overview

UniFuse (RAL+ICRA2021)

Office source code of paper UniFuse: Unidirectional Fusion for 360$^\circ$ Panorama Depth Estimation, arXiv, Demo

Preparation

Installation

Environments

python 3.6
Pytorch >= 1.0.0
CUDA >= 9.0

Install requirements

pip install -r requirements.txt

Datasets

Please download the preferred datasets, i.e., Matterport3D, Stanford2D3D, 3D60 and PanoSUNCG. For Matterport3D, please preprocess it following M3D-README.md.

Training

UniFuse on Matterport3D

python train.py --data_path $DATA_PATH \
-dataset matterport3d \
--model_name Matterport3D_UniFuse \
--batch_size 6 \
--num_epochs 100 \
--height 512 \
--width 1024 \
--imagenet_pretrained \
--net UniFuse

Equirectangular baseline on Matterport3D

python train.py --data_path $DATA_PATH \
-dataset matterport3d \
--model_name Matterport3D_Equi \
--batch_size 6 \
--num_epochs 100 \
--height 512 \
--width 1024 \
--imagenet_pretrained \
--net Equi

It is similar for other datasets.

Evaluation

Pre-trained models

The pre-trained models of UniFuse for 4 datasets are available, Matterport3D, Stanford2D3D, 3D60 and PanoSUNCG.

Test on a pre-trained model

python evaluate.py  --data_path $DATA_PATH --dataset matterport3d --load_weights_folder $MODEL_PATH

Citation

Please cite our paper if you find our work useful in your research.

@article{jiang2021unifuse,
      title={UniFuse: Unidirectional Fusion for 360$^{\circ}$ Panorama Depth Estimation}, 
      author={Hualie Jiang and Zhe Sheng and Siyu Zhu and Zilong Dong and Rui Huang},
	  journal={IEEE Robotics and Automation Letters},
	  year={2021},
	  publisher={IEEE}
}

Office source code of paper UniFuse: Unidirectional Fusion for 360$^\circ$ Panorama Depth Estimation

Related tags

Overview

UniFuse (RAL+ICRA2021)

Preparation

Installation

Datasets

Training

UniFuse on Matterport3D

Equirectangular baseline on Matterport3D

Evaluation

Pre-trained models

Test on a pre-trained model

Citation

Owner

Alibaba

MixRNet(Using mixup as regularization and tuning hyper-parameters for ResNets)

[TIP 2020] Multi-Temporal Scene Classification and Scene Change Detection with Correlation based Fusion

code for Multi-scale Matching Networks for Semantic Correspondence, ICCV

Meta Self-learning for Multi-Source Domain Adaptation： A Benchmark

Code for Talking Face Generation by Adversarially Disentangled Audio-Visual Representation (AAAI 2019)

Generate text captions for images from their CLIP embeddings. Includes PyTorch model code and example training script.

Unofficial pytorch implementation for Self-critical Sequence Training for Image Captioning. and others.

A PyTorch implementation of deep-learning-based registration

Implement the Pareto Optimizer and pcgrad to make a self-adaptive loss for multi-task

An easy way to build PyTorch datasets. Modularly build datasets and automatically cache processed results

The authors' implementation of Unsupervised Adversarial Learning of 3D Human Pose from 2D Joint Locations

Reproduces the results of the paper "Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations".

Real-time 3D multi-person detection made easy with OpenPose and the ZED

Multiview 3D object detection on MultiviewC dataset through moft3d.

A deep learning network built with TensorFlow and Keras to classify gender and estimate age.

This repository contains FEDOT - an open-source framework for automated modeling and machine learning (AutoML)

A code repository associated with the paper A Benchmark for Rough Sketch Cleanup by Chuan Yan, David Vanderhaeghe, and Yotam Gingold from SIGGRAPH Asia 2020.

IJCAI2020 & IJCV 2020 :city_sunrise: Unsupervised Scene Adaptation with Memory Regularization in vivo

PyTorch implementation of federated learning framework based on the acceleration of global momentum

This repository contains the code for the paper Neural RGB-D Surface Reconstruction