Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021

Last update: Nov 24, 2022

Overview

Geometric Vector Perceptron

Implementation of Geometric Vector Perceptron, a simple circuit with 3d rotation equivariance for learning over large biomolecules, in Pytorch. The repository may also contain experimentation to see if this could be easily extended to self-attention.

Install

$ pip install geometric-vector-perceptron

Functionality

GVP: Implementing the basic geometric vector perceptron.
GVPDropout: Adapted dropout for GVP in MPNN context
GVPLayerNorm: Adapted LayerNorm for GVP in MPNN context
GVP_MPNN: Adapted instance of Message Passing class from torch-geometric package. Still not tested.

Usage

import torch
from geometric_vector_perceptron import GVP

model = GVP(
    dim_vectors_in = 1024,
    dim_feats_in = 512,
    dim_vectors_out = 256,
    dim_feats_out = 512
)

feats, vectors = (torch.randn(1, 512), torch.randn(1, 1024, 3))

feats_out, vectors_out = model( (feats, vectors) ) # (1, 256), (1, 512, 3)

With the specialized dropout and layernorm as described in the paper

import torch
from torch import nn
from geometric_vector_perceptron import GVP, GVPDropout, GVPLayerNorm

model = GVP(
    dim_vectors_in = 1024,
    dim_feats_in = 512,
    dim_vectors_out = 256,
    dim_feats_out = 512
)

dropout = GVPDropout(0.2)
norm = GVPLayerNorm(512)

feats, vectors = (torch.randn(1, 512), torch.randn(1, 1024, 3))

feats, vectors = model( (feats, vectors) )
feats, vectors = dropout(feats, vectors)
feats, vectors = norm(feats, vectors)  # (1, 256), (1, 512, 3)

TF implementation:

The original implementation in TF by the paper authors can be found here: https://github.com/drorlab/gvp/

Citations

@inproceedings{
    anonymous2021learning,
    title={Learning from Protein Structure with Geometric Vector Perceptrons},
    author={Anonymous},
    booktitle={Submitted to International Conference on Learning Representations},
    year={2021},
    url={https://openreview.net/forum?id=1YLJDvSx6J4},
    note={under review}
}

Comments

Adds MPNN wrapper

Could you double check it?

I've tried to follow the original implementation ( https://github.com/drorlab/gvp/ see src/models.py ) but it's weird since they seem to mix the edge_attrs and the node features and return a mix of them, but later they seem to discard them so :man_shrugging: (whereas i only return node features - bc idk how to modify edge_attrs in an easy way -> the aggregation is always node-based)

opened by hypnopump 3
Fixes edge issues and distinguishes between covalent and proximity bonds
Gets all covalent bonds for a protein conformation (different than proximity)

Reduces unused computations

Fixes edges (now undirected graph)

Adapts denoising example accordingly (not in reconstruction example yet)
opened by hypnopump 0

PyTorch reimplementation of the Smooth ReLU activation function proposed in the paper "Real World Large Scale Recommendation Systems Reproducibility and Smooth Activations" [arXiv 2022].

Smooth ReLU in PyTorch Unofficial PyTorch reimplementation of the Smooth ReLU (SmeLU) activation function proposed in the paper Real World Large Scale

10 Jan 2, 2023

Seach Losses of our paper 'Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search', accepted by ICLR 2021.

CSE-Autoloss Designing proper loss functions for vision tasks has been a long-standing research direction to advance the capability of existing models

54 Dec 17, 2022

Neuralnetwork - Basic Multilayer Perceptron Neural Network for deep learning

Neural Network Just a basic Neural Network module Usage Example Importing Module

0 Nov 1, 2022

This repo contains the implementation of the algorithm proposed in Off-Belief Learning, ICML 2021.

Off-Belief Learning Introduction This repo contains the implementation of the algorithm proposed in Off-Belief Learning, ICML 2021. Environment Setup

32 Jan 5, 2023

From Perceptron model to Deep Neural Network from scratch in Python.

Neural-Network-Basics Aim of this Repository: From Perceptron model to Deep Neural Network (from scratch) in Python. ** Currently working on a basic N

1 Jan 14, 2022

[ICLR 2021, Spotlight] Large Scale Image Completion via Co-Modulated Generative Adversarial Networks

Large Scale Image Completion via Co-Modulated Generative Adversarial Networks, ICLR 2021 (Spotlight) Demo | Paper [NEW!] Time to play with our interac

373 Jan 2, 2023

A PyTorch implementation of Mugs proposed by our paper "Mugs: A Multi-Granular Self-Supervised Learning Framework".

Mugs: A Multi-Granular Self-Supervised Learning Framework This is a PyTorch implementation of Mugs proposed by our paper "Mugs: A Multi-Granular Self-

62 Nov 8, 2022

An attempt at the implementation of Glom, Geoffrey Hinton's new idea that integrates neural fields, predictive coding, top-down-bottom-up, and attention (consensus between columns)

GLOM - Pytorch (wip) An attempt at the implementation of Glom, Geoffrey Hinton's new idea that integrates neural fields, predictive coding,

173 Dec 14, 2022

Official Pytorch implementation of 6DRepNet: 6D Rotation representation for unconstrained head pose estimation.

6D Rotation Representation for Unconstrained Head Pose Estimation (Pytorch) Paper Thorsten Hempel and Ahmed A. Abdelrahman and Ayoub Al-Hamadi, "6D Ro

284 Dec 23, 2022

Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021

Related tags

Overview

Geometric Vector Perceptron

Install

Functionality

Usage

TF implementation:

Citations

You might also like...

PyTorch reimplementation of the Smooth ReLU activation function proposed in the paper "Real World Large Scale Recommendation Systems Reproducibility and Smooth Activations" [arXiv 2022].

Seach Losses of our paper 'Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search', accepted by ICLR 2021.

Neuralnetwork - Basic Multilayer Perceptron Neural Network for deep learning

This repo contains the implementation of the algorithm proposed in Off-Belief Learning, ICML 2021.

From Perceptron model to Deep Neural Network from scratch in Python.

[ICLR 2021, Spotlight] Large Scale Image Completion via Co-Modulated Generative Adversarial Networks

A PyTorch implementation of Mugs proposed by our paper "Mugs: A Multi-Granular Self-Supervised Learning Framework".

An attempt at the implementation of Glom, Geoffrey Hinton's new idea that integrates neural fields, predictive coding, top-down-bottom-up, and attention (consensus between columns)

Official Pytorch implementation of 6DRepNet: 6D Rotation representation for unconstrained head pose estimation.

Comments

Adds MPNN wrapper

Fixes edge issues and distinguishes between covalent and proximity bonds

Releases(0.0.14)

0.0.14(Jun 8, 2021)

0.0.12(Mar 7, 2021)

0.0.11(Mar 5, 2021)

0.0.10(Mar 3, 2021)

0.0.9(Mar 1, 2021)

0.0.8(Feb 24, 2021)

0.0.7(Feb 21, 2021)

0.0.6(Jan 27, 2021)

0.0.5(Jan 24, 2021)

0.0.3(Jan 24, 2021)

0.0.2(Jan 14, 2021)

0.0.1(Jan 13, 2021)

Owner

Phil Wang

A deep learning framework for historical document image analysis

A tool for making map images from OpenTTD save games

🔥🔥High-Performance Face Recognition Library on PaddlePaddle & PyTorch🔥🔥

OpenCVのGrabCut()を利用したセマンティックセグメンテーション向けアノテーションツール(Annotation tool using GrabCut() of OpenCV. It can be used to create datasets for semantic segmentation.)

A sketch extractor for anime/illustration.

Official repo for our 3DV 2021 paper "Monocular 3D Reconstruction of Interacting Hands via Collision-Aware Factorized Refinements".

TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction

Research code for the paper "How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models"

HODEmu, is both an executable and a python library that is based on Ragagnin 2021 in prep.

Revitalizing CNN Attention via Transformers in Self-Supervised Visual Representation Learning

Pytorch Implementation of Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic

GNN-based Recommendation Benchma

Implementation of our paper "DMT: Dynamic Mutual Training for Semi-Supervised Learning"

Official git repo for the CHIRP project

Easily pull telemetry data and create beautiful visualizations for analysis.

A large-scale video dataset for the training and evaluation of 3D human pose estimation models

Detector for Log4Shell exploitation attempts

The 1st Place Solution of the Facebook AI Image Similarity Challenge (ISC21) : Descriptor Track.

7th place solution of Human Protein Atlas - Single Cell Classification on Kaggle

[ICCV 2021] A Simple Baseline for Semi-supervised Semantic Segmentation with Strong Data Augmentation