BRepNet: A topological message passing system for solid models

Overview

BRepNet: A topological message passing system for solid models

This repository contains the an implementation of BRepNet: A topological message passing system for solid models.

BRepNet kernel image

About BRepNet

BRepNet is a neural network specifically designed to operate on solid models. It uses additional topological information present in the boundary representation (B-Rep) data structure to perform convolutions in a way which is not possible for arbitrary graphs. As B-Reps describe manifolds, they contain additional topological information which includes the ordering of edges around faces as well as the face adjacency. The topology is defined using oriented edges called coedges. Each coedge maintains an adjacency relationship with the next and previous coedge around its parent face, the mating coedge on the adjacent face, the parent face and the parent edge.

B-Rep topology and topological walks

Using this information, we can identify faces, edges and coedges in the neighborhood of some starting coedge (red), using topological walks. A topological walk is a series of instructions we move us from the starting coedge to a nearby entity. In the figure above (B) the we show a walk from the red starting coedge to its mating coedge, to the next coedge in the loop, to mating coedge and finally to the parent face. Using multiple topological walks we can define a group of entities in the neighborhood of the starting coedge. The instructions which define the neighboring entities are marked in the figure (C). The BRepNet implementation allows you to define any group of entities using a kernel file. See here for an example of a kernel file for kernel entities shown above.

Convolution

The BRepNet convolution algorithm concatenates feature vectors from the entities defined in the kernel file relative to the starting coedge (red). The resulting vector is passed through an MLP and the output becomes the hidden state for this coedge in the next network layer. The procedure is repeated for each coedge in the model, then new hidden state vectors for the faces and edges are generated by pooling the coedge hidden states onto their parent faces and edges. See the paper for more details. The actual implementation of the BRepNet convolution can been seen in the BRepNetLayer.forward() method.

Citing this work

@inproceedings{lambourne2021brepnet,
 title = {BRepNet: A Topological Message Passing System for Solid Models},
 author = {Joseph G. Lambourne and Karl D.D. Willis and Pradeep Kumar Jayaraman and Aditya Sanghi and Peter Meltzer and Hooman Shayani},
 eprint = {2104.00706},
 eprinttype = {arXiv},
 eprintclass = {cs.LG},
 booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
 year = {2021}
}

Quickstart

Setting up the environment

git clone https://github.com/AutodeskAILab/BRepNet.git
cd BRepNet
conda env create -f environment.yml
conda activate brepnet

For GPU training you will need to change the pytorch install to include your cuda version. i.e.

conda install pytorch cudatoolkit=11.1 -c pytorch -c conda-forge

For training with multiple workers you may hit errors of the form OSError: [Errno 24] Too many open files. In this case you need to increase the number of available file handles on the machine using

ulimit -Sn 10000

I find I need to set the limit to 10000 for 10 worker threads.

Download the dataset

You can download the step distribution of the Fusion 360 Gallery segmentation dataset from this link. The zip is 3.2Gb. Alternatively download using curl

cd /path/to/where_you_keep_data/
curl https://fusion-360-gallery-dataset.s3-us-west-2.amazonaws.com/segmentation/s2.0.0/s2.0.0.zip -o s2.0.0.zip
unzip s2.0.0.zip

If you are interested in building your own dataset using other step files then the procedure is documented here

Processing the STEP data

Run the quickstart script to extract topology and geometry information from the step data ready to train the network.

cd BRepNet/
python -m pipeline.quickstart --dataset_dir /path/to/where_you_keep_data/s2.0.0 --num_workers 5

This may take up to 10 minutes to complete.

Training the model

You are then ready to train the model. The quickstart script should exit telling you a default command to use which should be something like

python -m train.train \
  --dataset_file /path/to/where_you_keep_data/s2.0.0/processed/dataset.json \
  --dataset_dir  /path/to/where_you_keep_data/s2.0.0/processed/ \
  --max_epochs 50

You may want to adjust the --num_workers and --gpus parameters to match your machine. The model runs with the pytorch-lightning ddp-spawn mode, so you can choose either 1 worker thread and multiple gpus or multiple threads and a single gpu. The options and hyper-parameters for BRepNet can be seen in BRepNet.add_model_specific_args in brepnet.py. For a full list of all hyper-parameters including those defined in pytorch-lightning see

python -m train.train --help

Monitoring the loss, accuracy and IoU

By default BRepNet will log data to tensorboard in a folder called logs. Each time you run the model the logs will be placed in a separate folder inside the logs directory with paths based on the date and time. At the start of training the path to the log folder will be printed into the shell. To monitory the process you can use

cd BRepNet
tensorboard --logdir logs

A trained model is also saved every time the validation loss reaches a minimum. The model will be in the same folder as the tensorboard logs

./logs/<date>/<time>/checkpoints

Testing the network

python -m eval.test \
  --dataset_file /path/to/dataset_file.json \
  --dataset_dir /path/to/data_dir \
  --model BRepNet/logs/<day>/<time>/checkpoints/epoch=x-step=x.ckpt

Visualizing the segmentation data

You can visualize the segmentation data using a Jupyter notebook and the tools in the visualization folder. An example of how to view the segmentation information in the dataset is here.

Evaluating the segmentation on your own STEP data

To evaluate the model on you own step data you can use the script evaluate_folder.py

python -m eval.evaluate_folder  \
  --dataset_dir ./example_files/step_examples
  --dataset_file ./example_files/feature_standardization/s2.0.0_step_all_features.json \
  --model ./example_files/pretrained_models/pretrained_s2.0.0_step_all_features_0519_073100.ckpt

This will loop over all step or stp files in ./example_files/step_examples and create "logits" files in example_files/step_examples/temp_working/logits. The logits files contain one row for each face in the step data. The columns give the probabilities that the corresponding face belongs to a given segment.

The notebook find_and_display_segmentation.ipynb runs through the entire process of evaluating the model and displaying the predicted segmentation.

Running the tests

If you need to run the tests then this can be done using

python -m unittest

The new data-pipeline based on Open Cascade

The original BRepNet pipeline used proprietary code to process data from solid models and convert these to network input. In an effort to make this BRepNet as reusable as possible we have converted this pipeline to work with Open Cascade and python OCC. As with any kind of translation between solid model formats, the translation to step introduces some differences in the data. These are documented here. When training with the default options given above you will obtain very similar numbers to the ones published.

License

Shield: CC BY-NC-SA 4.0

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

CC BY-NC-SA 4.0

Owner
Autodesk AI Lab
Autodesk AI Lab
PyTorch implementation of TSception V2 using DEAP dataset

TSception This is the PyTorch implementation of TSception V2 using DEAP dataset in our paper: Yi Ding, Neethu Robinson, Su Zhang, Qiuhao Zeng, Cuntai

Yi Ding 27 Dec 15, 2022
A DCGAN to generate anime faces using custom mined dataset

Anime-Face-GAN-Keras A DCGAN to generate anime faces using custom dataset in Keras. Dataset The dataset is created by crawling anime database websites

Pavitrakumar P 190 Jan 03, 2023
Official implementation of "Robust channel-wise illumination estimation"

This repository provides the official implementation of "Robust channel-wise illumination estimation." accepted in BMVC (2021).

Firas Laakom 4 Nov 08, 2022
Official PyTorch implementation of DD3D: Is Pseudo-Lidar needed for Monocular 3D Object detection? (ICCV 2021), Dennis Park*, Rares Ambrus*, Vitor Guizilini, Jie Li, and Adrien Gaidon.

DD3D: "Is Pseudo-Lidar needed for Monocular 3D Object detection?" Install // Datasets // Experiments // Models // License // Reference Full video Offi

Toyota Research Institute - Machine Learning 364 Dec 27, 2022
AIR^2 for Interaction Prediction

This is the repository for AIR^2 for Interaction Prediction. Explanation of the solution: Video: link License AIR is released under the Apache 2.0 lic

21 Sep 27, 2022
Jarvis Project is a basic virtual assistant that uses TensorFlow for learning.

Jarvis_proyect Jarvis Project is a basic virtual assistant that uses TensorFlow for learning. Latest version 0.1 Features: Good morning protocol Tell

Anze Kovac 3 Aug 31, 2022
This repo holds code for TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation

TransUNet This repo holds code for TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation Usage

1.4k Jan 04, 2023
Citation Intent Classification in scientific papers using the Scicite dataset an Pytorch

Citation Intent Classification Table of Contents About the Project Built With Installation Usage Acknowledgments About The Project Citation Intent Cla

Federico Nocentini 4 Mar 04, 2022
MonoRCNN is a monocular 3D object detection method for automonous driving

MonoRCNN MonoRCNN is a monocular 3D object detection method for automonous driving, published at ICCV 2021. This project is an implementation of MonoR

87 Dec 27, 2022
Single-Stage 6D Object Pose Estimation, CVPR 2020

Overview This repository contains the code for the paper Single-Stage 6D Object Pose Estimation. Yinlin Hu, Pascal Fua, Wei Wang and Mathieu Salzmann.

CVLAB @ EPFL 89 Dec 26, 2022
Official implementation for the paper "Attentive Prototypes for Source-free Unsupervised Domain Adaptive 3D Object Detection"

Attentive Prototypes for Source-free Unsupervised Domain Adaptive 3D Object Detection PyTorch code release of the paper "Attentive Prototypes for Sour

Deepti Hegde 23 Oct 17, 2022
PSML: A Multi-scale Time-series Dataset for Machine Learning in Decarbonized Energy Grids

PSML: A Multi-scale Time-series Dataset for Machine Learning in Decarbonized Energy Grids The electric grid is a key enabling infrastructure for the a

Texas A&M Engineering Research 19 Jan 07, 2023
Implementations of paper Controlling Directions Orthogonal to a Classifier

Classifier Orthogonalization Implementations of paper Controlling Directions Orthogonal to a Classifier , ICLR 2022, Yilun Xu, Hao He, Tianxiao Shen,

Yilun Xu 33 Dec 01, 2022
Safe Model-Based Reinforcement Learning using Robust Control Barrier Functions

README Repository containing the code for the paper "Safe Model-Based Reinforcement Learning using Robust Control Barrier Functions". Specifically, an

Yousef Emam 13 Nov 24, 2022
PyTorch implementation for paper Neural Marching Cubes.

NMC PyTorch implementation for paper Neural Marching Cubes, Zhiqin Chen, Hao Zhang. Paper | Supplementary Material (to be updated) Citation If you fin

Zhiqin Chen 109 Dec 27, 2022
Code for testing convergence rates of Lipschitz learning on graphs

📈 LipschitzLearningRates The code in this repository reproduces the experimental results on convergence rates for k-nearest neighbor graph infinity L

2 Dec 20, 2021
Optimizing synthesizer parameters using gradient approximation

Optimizing synthesizer parameters using gradient approximation NASH 2021 Hackathon! These are some experiments I conducted during NASH 2021, the Neura

Jordie Shier 10 Feb 10, 2022
Implementation of "Bidirectional Projection Network for Cross Dimension Scene Understanding" CVPR 2021 (Oral)

Bidirectional Projection Network for Cross Dimension Scene Understanding CVPR 2021 (Oral) [ Project Webpage ] [ arXiv ] [ Video ] Existing segmentatio

Hu Wenbo 135 Dec 26, 2022
DeLighT: Very Deep and Light-Weight Transformers

DeLighT: Very Deep and Light-weight Transformers This repository contains the source code of our work on building efficient sequence models: DeFINE (I

Sachin Mehta 440 Dec 18, 2022