Code for Iso-Points: Optimizing Neural Implicit Surfaces with Hybrid Representations

Overview

Implementation for Iso-Points (CVPR 2021)

Official code for paper Iso-Points: Optimizing Neural Implicit Surfaces with Hybrid Representations

paper | supplementary material | project page

Overview

Iso-points are well-distributed points which lie on the neural iso-surface, they are an explicit form of representation of the implicit surfaces. We propose using iso-points to augment the optimization of implicit neural surfaces. The implicit and explicit surface representations are coupled, i.e. the implicit model determines the locations and normals of iso-points, whereas the iso-points can be utilized to control the optimization of the implicit model.

The implementation of the key steps for iso-points extraction is in levelset_sampling.py and utils/point_processing.py. To demonstrate the utilisation of iso-points, we provide scripts for multiple applications and scenarios:

Demo

Installation

This code is built as an extension of out Differentiable Surface Splatting pytorch library (DSS), which depends on pytorch3d, torch_cluster. Currently we support up to pytorch 1.6.

git clone --recursive https://github.com/yifita/iso-points.git
cd iso-points

# conda environment and dependencies
# update conda
conda update -n base -c defaults conda
# install requirements
conda env create --name DSS -f environment.yml
conda activate DSS

# build additional dependencies of DSS
# FRNN - fixed radius nearest neighbors
cd external/FRNN/external
git submodule update --init --recursive
cd prefix_sum
python setup.py install
cd ../..
python setup.py install

# build batch-svd
cd ../torch-batch-svd
python setup.py install

# build DSS itself
cd ../..
python setup.py develop

prepare data

Download data

cd data
wget https://igl.ethz.ch/projects/iso-points/data.zip
unzip data.zip
rm data.zip

Including subset of masked DTU data (courtesy of Yariv et.al.), synthetic rendered multiview data, and masked furu stereo reconstruction of DTU dataset.

multiview reconstruction

sampling-with-iso-points

# train baseline implicit representation only using ray-tracing
python train_mvr.py configs/compressor_implicit.yml --exit-after 6000

# train with uniform iso-points
python train_mvr.py configs/compressor_uni.yml --exit-after 6000

# train with iso-points distributed according to loss value (hard example mining)
python train_mvr.py configs/compressor_uni_lossS.yml --exit-after 6000

sampling result

DTU-data

python train_mvr.py configs/dtu55_iso.yml

dtu mvr result

implicit surface to noisy point cloud

python test_dtu_points.py data/DTU_furu/scan122.ply --use_off_normal_loss -o exp/points_3d_outputs/scan122_ours

cite

Please cite us if you find the code useful!

@inproceedings{yifan2020isopoints,
      title={Iso-Points: Optimizing Neural Implicit Surfaces with Hybrid Representations},
      author={Wang Yifan and Shihao Wu and Cengiz Oztireli and Olga Sorkine-Hornung},
      year={2020},
      booktitle = {CVPR},
      year = {2020},
}

Acknowledgement

We would like to thank Viviane Yang for her help with the point2surf code. This work was supported in parts by Apple scholarship, SWISSHEART Failure Network (SHFN), and UKRI Future Leaders Fellowship [grant number MR/T043229/1]

Owner
Yifan Wang
PhD student @ ETH Zurich
Yifan Wang
PyTorch implementation of "LayoutTransformer: Layout Generation and Completion with Self-attention"

PyTorch implementation of "LayoutTransformer: Layout Generation and Completion with Self-attention" to appear in ICCV 2021

Kamal Gupta 75 Dec 23, 2022
A lightweight Python-based 3D network multi-agent simulator. Uses a cell-based congestion model. Calculates risk, loudness and battery capacities of the agents. Suitable for 3D network optimization tasks.

AMAZ3DSim AMAZ3DSim is a lightweight python-based 3D network multi-agent simulator. It uses a cell-based congestion model. It calculates risk, battery

Daniel Hirsch 13 Nov 04, 2022
A Python Library for Graph Outlier Detection (Anomaly Detection)

PyGOD is a Python library for graph outlier detection (anomaly detection). This exciting yet challenging field has many key applications, e.g., detect

PyGOD Team 757 Jan 04, 2023
Twins: Revisiting the Design of Spatial Attention in Vision Transformers

Twins: Revisiting the Design of Spatial Attention in Vision Transformers Very recently, a variety of vision transformer architectures for dense predic

482 Dec 18, 2022
MassiveSumm: a very large-scale, very multilingual, news summarisation dataset

MassiveSumm: a very large-scale, very multilingual, news summarisation dataset This repository contains links to data and code to fetch and reproduce

Daniel Varab 19 Dec 16, 2022
CondNet: Conditional Classifier for Scene Segmentation

CondNet: Conditional Classifier for Scene Segmentation Introduction The fully convolutional network (FCN) has achieved tremendous success in dense vis

ycszen 31 Jul 22, 2022
Latte: Cross-framework Python Package for Evaluation of Latent-based Generative Models

Cross-framework Python Package for Evaluation of Latent-based Generative Models Latte Latte (for LATent Tensor Evaluation) is a cross-framework Python

Karn Watcharasupat 30 Sep 08, 2022
Official repo for QHack—the quantum machine learning hackathon

Note: This repository has been frozen while we consider the submissions for the QHack Open Hackathon. We hope you enjoyed the event! Welcome to QHack,

Xanadu 118 Jan 05, 2023
CCP dataset from Clothing Co-Parsing by Joint Image Segmentation and Labeling

Clothing Co-Parsing (CCP) Dataset Clothing Co-Parsing (CCP) dataset is a new clothing database including elaborately annotated clothing items. 2, 098

Wei Yang 434 Dec 24, 2022
SSD-based Object Detection in PyTorch

SSD-based Object Detection in PyTorch 서강대학교 현대모비스 SW 프로그램에서 진행한 인공지능 프로젝트입니다. Jetson nano를 이용해 pre-trained network를 fine tuning시켜 차량 및 신호등 인식을 구현하였습니다

Haneul Kim 1 Nov 16, 2021
Contrastive unpaired image-to-image translation, faster and lighter training than cyclegan (ECCV 2020, in PyTorch)

Contrastive Unpaired Translation (CUT) video (1m) | video (10m) | website | paper We provide our PyTorch implementation of unpaired image-to-image tra

1.7k Dec 27, 2022
A PyTorch implementation of "Semi-Supervised Graph Classification: A Hierarchical Graph Perspective" (WWW 2019)

SEAL ⠀⠀⠀ A PyTorch implementation of Semi-Supervised Graph Classification: A Hierarchical Graph Perspective (WWW 2019) Abstract Node classification an

Benedek Rozemberczki 202 Dec 27, 2022
Manipulation OpenAI Gym environments to simulate robots at the STARS lab

Manipulator Learning This repository contains a set of manipulation environments that are compatible with OpenAI Gym and simulated in pybullet. In par

STARS Laboratory 5 Dec 08, 2022
Code & Data for the Paper "Time Masking for Temporal Language Models", WSDM 2022

Time Masking for Temporal Language Models This repository provides a reference implementation of the paper: Time Masking for Temporal Language Models

Guy Rosin 12 Jan 06, 2023
This is a Pytorch implementation of the paper: Self-Supervised Graph Transformer on Large-Scale Molecular Data.

This is a Pytorch implementation of the paper: Self-Supervised Graph Transformer on Large-Scale Molecular Data.

212 Dec 25, 2022
Official Repo for Ground-aware Monocular 3D Object Detection for Autonomous Driving

Visual 3D Detection Package: This repo aims to provide flexible and reproducible visual 3D detection on KITTI dataset. We expect scripts starting from

Yuxuan Liu 305 Dec 19, 2022
PyTorch implementation of the WarpedGANSpace: Finding non-linear RBF paths in GAN latent space (ICCV 2021)

Authors official PyTorch implementation of the "WarpedGANSpace: Finding non-linear RBF paths in GAN latent space" [ICCV 2021].

Christos Tzelepis 100 Dec 06, 2022
Repo for WWW 2022 paper: Progressively Optimized Bi-Granular Document Representation for Scalable Embedding Based Retrieval

BiDR Repo for WWW 2022 paper: Progressively Optimized Bi-Granular Document Representation for Scalable Embedding Based Retrieval. Requirements torch==

Microsoft 11 Oct 20, 2022
SEAN: Image Synthesis with Semantic Region-Adaptive Normalization (CVPR 2020, Oral)

SEAN: Image Synthesis with Semantic Region-Adaptive Normalization (CVPR 2020 Oral) Figure: Face image editing controlled via style images and segmenta

Peihao Zhu 579 Dec 30, 2022
Problem-943.-ACMP - Problem 943. ACMP

Problem-943.-ACMP В "main.py" расположен вариант моего решения задачи 943 с серв

Konstantin Dyomshin 2 Aug 19, 2022