Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021

Last update: Nov 24, 2022

Overview

Geometric Vector Perceptron

Implementation of Geometric Vector Perceptron, a simple circuit with 3d rotation equivariance for learning over large biomolecules, in Pytorch. The repository may also contain experimentation to see if this could be easily extended to self-attention.

Install

$ pip install geometric-vector-perceptron

Functionality

GVP: Implementing the basic geometric vector perceptron.
GVPDropout: Adapted dropout for GVP in MPNN context
GVPLayerNorm: Adapted LayerNorm for GVP in MPNN context
GVP_MPNN: Adapted instance of Message Passing class from torch-geometric package. Still not tested.

Usage

import torch
from geometric_vector_perceptron import GVP

model = GVP(
    dim_vectors_in = 1024,
    dim_feats_in = 512,
    dim_vectors_out = 256,
    dim_feats_out = 512
)

feats, vectors = (torch.randn(1, 512), torch.randn(1, 1024, 3))

feats_out, vectors_out = model( (feats, vectors) ) # (1, 256), (1, 512, 3)

With the specialized dropout and layernorm as described in the paper

import torch
from torch import nn
from geometric_vector_perceptron import GVP, GVPDropout, GVPLayerNorm

model = GVP(
    dim_vectors_in = 1024,
    dim_feats_in = 512,
    dim_vectors_out = 256,
    dim_feats_out = 512
)

dropout = GVPDropout(0.2)
norm = GVPLayerNorm(512)

feats, vectors = (torch.randn(1, 512), torch.randn(1, 1024, 3))

feats, vectors = model( (feats, vectors) )
feats, vectors = dropout(feats, vectors)
feats, vectors = norm(feats, vectors)  # (1, 256), (1, 512, 3)

TF implementation:

The original implementation in TF by the paper authors can be found here: https://github.com/drorlab/gvp/

Citations

@inproceedings{
    anonymous2021learning,
    title={Learning from Protein Structure with Geometric Vector Perceptrons},
    author={Anonymous},
    booktitle={Submitted to International Conference on Learning Representations},
    year={2021},
    url={https://openreview.net/forum?id=1YLJDvSx6J4},
    note={under review}
}

Comments

Adds MPNN wrapper

Could you double check it?

I've tried to follow the original implementation ( https://github.com/drorlab/gvp/ see src/models.py ) but it's weird since they seem to mix the edge_attrs and the node features and return a mix of them, but later they seem to discard them so :man_shrugging: (whereas i only return node features - bc idk how to modify edge_attrs in an easy way -> the aggregation is always node-based)

opened by hypnopump 3
Fixes edge issues and distinguishes between covalent and proximity bonds
Gets all covalent bonds for a protein conformation (different than proximity)

Reduces unused computations

Fixes edges (now undirected graph)

Adapts denoising example accordingly (not in reconstruction example yet)
opened by hypnopump 0

PyTorch reimplementation of the Smooth ReLU activation function proposed in the paper "Real World Large Scale Recommendation Systems Reproducibility and Smooth Activations" [arXiv 2022].

Smooth ReLU in PyTorch Unofficial PyTorch reimplementation of the Smooth ReLU (SmeLU) activation function proposed in the paper Real World Large Scale

10 Jan 2, 2023

Seach Losses of our paper 'Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search', accepted by ICLR 2021.

CSE-Autoloss Designing proper loss functions for vision tasks has been a long-standing research direction to advance the capability of existing models

54 Dec 17, 2022

Neuralnetwork - Basic Multilayer Perceptron Neural Network for deep learning

Neural Network Just a basic Neural Network module Usage Example Importing Module

0 Nov 1, 2022

This repo contains the implementation of the algorithm proposed in Off-Belief Learning, ICML 2021.

Off-Belief Learning Introduction This repo contains the implementation of the algorithm proposed in Off-Belief Learning, ICML 2021. Environment Setup

32 Jan 5, 2023

From Perceptron model to Deep Neural Network from scratch in Python.

Neural-Network-Basics Aim of this Repository: From Perceptron model to Deep Neural Network (from scratch) in Python. ** Currently working on a basic N

1 Jan 14, 2022

[ICLR 2021, Spotlight] Large Scale Image Completion via Co-Modulated Generative Adversarial Networks

Large Scale Image Completion via Co-Modulated Generative Adversarial Networks, ICLR 2021 (Spotlight) Demo | Paper [NEW!] Time to play with our interac

373 Jan 2, 2023

A PyTorch implementation of Mugs proposed by our paper "Mugs: A Multi-Granular Self-Supervised Learning Framework".

Mugs: A Multi-Granular Self-Supervised Learning Framework This is a PyTorch implementation of Mugs proposed by our paper "Mugs: A Multi-Granular Self-

62 Nov 8, 2022

An attempt at the implementation of Glom, Geoffrey Hinton's new idea that integrates neural fields, predictive coding, top-down-bottom-up, and attention (consensus between columns)

GLOM - Pytorch (wip) An attempt at the implementation of Glom, Geoffrey Hinton's new idea that integrates neural fields, predictive coding,

173 Dec 14, 2022

Official Pytorch implementation of 6DRepNet: 6D Rotation representation for unconstrained head pose estimation.

6D Rotation Representation for Unconstrained Head Pose Estimation (Pytorch) Paper Thorsten Hempel and Ahmed A. Abdelrahman and Ayoub Al-Hamadi, "6D Ro

284 Dec 23, 2022

Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021

Related tags

Overview

Geometric Vector Perceptron

Install

Functionality

Usage

TF implementation:

Citations

You might also like...

PyTorch reimplementation of the Smooth ReLU activation function proposed in the paper "Real World Large Scale Recommendation Systems Reproducibility and Smooth Activations" [arXiv 2022].

Seach Losses of our paper 'Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search', accepted by ICLR 2021.

Neuralnetwork - Basic Multilayer Perceptron Neural Network for deep learning

This repo contains the implementation of the algorithm proposed in Off-Belief Learning, ICML 2021.

From Perceptron model to Deep Neural Network from scratch in Python.

[ICLR 2021, Spotlight] Large Scale Image Completion via Co-Modulated Generative Adversarial Networks

A PyTorch implementation of Mugs proposed by our paper "Mugs: A Multi-Granular Self-Supervised Learning Framework".

An attempt at the implementation of Glom, Geoffrey Hinton's new idea that integrates neural fields, predictive coding, top-down-bottom-up, and attention (consensus between columns)

Official Pytorch implementation of 6DRepNet: 6D Rotation representation for unconstrained head pose estimation.

Comments

Adds MPNN wrapper

Fixes edge issues and distinguishes between covalent and proximity bonds

Releases(0.0.14)

0.0.14(Jun 8, 2021)

0.0.12(Mar 7, 2021)

0.0.11(Mar 5, 2021)

0.0.10(Mar 3, 2021)

0.0.9(Mar 1, 2021)

0.0.8(Feb 24, 2021)

0.0.7(Feb 21, 2021)

0.0.6(Jan 27, 2021)

0.0.5(Jan 24, 2021)

0.0.3(Jan 24, 2021)

0.0.2(Jan 14, 2021)

0.0.1(Jan 13, 2021)

Owner

Phil Wang

Element selection for functional materials discovery by integrated machine learning of atomic contributions to properties

Reproduces ResNet-V3 with pytorch

Anonymize BLM Protest Images

[ACM MM 2019 Oral] Cycle In Cycle Generative Adversarial Networks for Keypoint-Guided Image Generation

FPGA: Fast Patch-Free Global Learning Framework for Fully End-to-End Hyperspectral Image Classification

[CVPR'21] Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration

Model search is a framework that implements AutoML algorithms for model architecture search at scale

traiNNer is an open source image and video restoration (super-resolution, denoising, deblurring and others) and image to image translation toolbox based on PyTorch.

Facial Action Unit Intensity Estimation via Semantic Correspondence Learning with Dynamic Graph Convolution

The pytorch implementation of SOKD (BMVC2021).

Shared Attention for Multi-label Zero-shot Learning

TensorFlow tutorials and best practices.

STYLER: Style Factor Modeling with Rapidity and Robustness via Speech Decomposition for Expressive and Controllable Neural Text to Speech

The PyTorch re-implement of a 3D CNN Tracker to extract coronary artery centerlines with state-of-the-art (SOTA) performance. (paper: 'Coronary artery centerline extraction in cardiac CT angiography using a CNN-based orientation classiﬁer')

Source code for "Pack Together: Entity and Relation Extraction with Levitated Marker"

The Generic Manipulation Driver Package - Implements a ROS Interface over the robotics toolbox for Python

Source code and notebooks to reproduce experiments and benchmarks on Bias Faces in the Wild (BFW).

The PASS dataset: pretrained models and how to get the data - PASS: Pictures without humAns for Self-Supervised Pretraining

ML-PersonalWork - Big assignment PersonalWork in Machine Learning, 2021 autumn BUAA.

A PyTorch Implementation of "Neural Arithmetic Logic Units"