A library for optimization on Riemannian manifolds

Overview

TensorFlow RiemOpt

PyPI arXiv Build Status Coverage Status Code style: black License

A library for manifold-constrained optimization in TensorFlow.

Installation

To install the latest development version from GitHub:

pip install git+https://github.com/master/tensorflow-riemopt.git

To install a package from PyPI:

pip install tensorflow-riemopt

Features

The core package implements concepts in differential geometry, such as manifolds and Riemannian metrics with associated exponential and logarithmic maps, geodesics, retractions, and transports. For manifolds, where closed-form expressions are not available, the library provides numerical approximations.

import tensorflow_riemopt as riemopt

S = riemopt.manifolds.Sphere()

x = S.projx(tf.constant([0.1, -0.1, 0.1]))
u = S.proju(x, tf.constant([1., 1., 1.]))
v = S.proju(x, tf.constant([-0.7, -1.4, 1.4]))

y = S.exp(x, v)

u_ = S.transp(x, y, u)
v_ = S.transp(x, y, v)

Manifolds

  • manifolds.Cholesky - manifold of lower triangular matrices with positive diagonal elements
  • manifolds.Euclidian - unconstrained manifold with the Euclidean metric
  • manifolds.Grassmannian - manifold of p-dimensional linear subspaces of the n-dimensional space
  • manifolds.Hyperboloid - manifold of n-dimensional hyperbolic space embedded in the n+1-dimensional Minkowski space
  • manifolds.Poincare - the Poincaré ball model of the hyperbolic space
  • manifolds.Product - Cartesian product of manifolds
  • manifolds.SPDAffineInvariant - manifold of symmetric positive definite (SPD) matrices endowed with the affine-invariant metric
  • manifolds.SPDLogCholesky - SPD manifold with the Log-Cholesky metric
  • manifolds.SPDLogEuclidean - SPD manifold with the Log-Euclidean metric
  • manifolds.SpecialOrthogonal - manifold of rotation matrices
  • manifolds.Sphere - manifold of unit-normalized points
  • manifolds.StiefelEuclidean - manifold of orthonormal p-frames in the n-dimensional space endowed with the Euclidean metric
  • manifolds.StiefelCanonical - Stiefel manifold with the canonical metric
  • manifolds.StiefelCayley - Stiefel manifold the retraction map via an iterative Cayley transform

Optimizers

Constrained optimization algorithms work as drop-in replacements for Keras optimizers for sparse and dense updates in both Eager and Graph modes.

  • optimizers.RiemannianSGD - Riemannian Gradient Descent
  • optimizers.RiemannianAdam - Riemannian Adam and AMSGrad
  • optimizers.ConstrainedRMSProp - Constrained RMSProp

Layers

  • layers.ManifoldEmbedding - constrained keras.layers.Embedding layer

Examples

  • SPDNet - Huang, Zhiwu, and Luc Van Gool. "A Riemannian network for SPD matrix learning." Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. AAAI Press, 2017.
  • LieNet - Huang, Zhiwu, et al. "Deep learning on Lie groups for skeleton-based action recognition." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
  • GrNet - Huang, Zhiwu, Jiqing Wu, and Luc Van Gool. "Building Deep Networks on Grassmann Manifolds." AAAI. AAAI Press, 2018.
  • Hyperbolic Neural Network - Ganea, Octavian, Gary Bécigneul, and Thomas Hofmann. "Hyperbolic neural networks." Advances in neural information processing systems. 2018.
  • Poincaré GloVe - Tifrea, Alexandru, Gary Becigneul, and Octavian-Eugen Ganea. "Poincaré Glove: Hyperbolic Word Embeddings." International Conference on Learning Representations. 2018.

References

If you find TensorFlow RiemOpt useful in your research, please cite:

@misc{smirnov2021tensorflow,
      title={TensorFlow RiemOpt: a library for optimization on Riemannian manifolds},
      author={Oleg Smirnov},
      year={2021},
      eprint={2105.13921},
      archivePrefix={arXiv},
      primaryClass={cs.MS}
}

Acknowledgment

TensorFlow RiemOpt was inspired by many similar projects:

  • Manopt, a matlab toolbox for optimization on manifolds
  • Pymanopt, a Python toolbox for optimization on manifolds
  • Geoopt: Riemannian Optimization in PyTorch
  • Geomstats, an open-source Python package for computations and statistics on nonlinear manifolds

License

The code is MIT-licensed.

You might also like...
Distributed Asynchronous Hyperparameter Optimization better than HyperOpt.
Distributed Asynchronous Hyperparameter Optimization better than HyperOpt.

UltraOpt : Distributed Asynchronous Hyperparameter Optimization better than HyperOpt. UltraOpt is a simple and efficient library to minimize expensive

Official code for paper "Optimization for Oriented Object Detection via Representation Invariance Loss".

Optimization for Oriented Object Detection via Representation Invariance Loss By Qi Ming, Zhiqiang Zhou, Lingjuan Miao, Xue Yang, and Yunpeng Dong. Th

Keras + Hyperopt: A very simple wrapper for convenient hyperparameter optimization

This project is now archived. It's been fun working on it, but it's time for me to move on. Thank you for all the support and feedback over the last c

Bayesian optimization in PyTorch

BoTorch is a library for Bayesian Optimization built on PyTorch. BoTorch is currently in beta and under active development! Why BoTorch ? BoTorch Prov

optimization routines for hyperparameter tuning
optimization routines for hyperparameter tuning

Optunity is a library containing various optimizers for hyperparameter tuning. Hyperparameter tuning is a recurrent problem in many machine learning t

Distributed Asynchronous Hyperparameter Optimization in Python

Hyperopt: Distributed Hyperparameter Optimization Hyperopt is a Python library for serial and parallel optimization over awkward search spaces, which

Hyper-parameter optimization for sklearn

hyperopt-sklearn Hyperopt-sklearn is Hyperopt-based model selection among machine learning algorithms in scikit-learn. See how to use hyperopt-sklearn

A Python implementation of global optimization with gaussian processes.
A Python implementation of global optimization with gaussian processes.

Bayesian Optimization Pure Python implementation of bayesian global optimization with gaussian processes. PyPI (pip): $ pip install bayesian-optimizat

Safe Bayesian Optimization
Safe Bayesian Optimization

SafeOpt - Safe Bayesian Optimization This code implements an adapted version of the safe, Bayesian optimization algorithm, SafeOpt [1], [2]. It also p

Comments
  • Projection on SPDs is not projecting onto SPDs

    Projection on SPDs is not projecting onto SPDs

    Hi, nice to see another package doing optimizationon manifolds! I have not yet had the time to check this versus what pymanopt is doing (I think they use tensor flow as a backend, too?) But I just noticed that

    https://github.com/master/tensorflow-manopt/blob/93402f6770d5b3c45f232340fddfa92a7126f19a/tensorflow_manopt/manifolds/symmetric_positive.py#L37-L41

    This might be wrong. For SPDs, the characteristic property is, that all eigenvalues are positive, so this projection is not projection onto the manifold (of SPDs) but onto the set of positive semidefinite matrices. There is no projection onto the SPDs since that set is open in the set of (symmetric) matrices.

    opened by kellertuer 2
  • GrNet produces NaN entries in input tensor

    GrNet produces NaN entries in input tensor

    Hi! First of all, really appreciate you guys taking the time to build a much required riemmannian geometry based package in tensorflow. It is proving to be quite useful for me. However, I recently ran the [GrNet code] (https://github.com/master/tensorflow-riemopt/tree/master/examples/grnet) with the AFEW dataset(the default dataset used in the code) on my machine and it seems at some point the input tensors get filled with NaN values. I tried tinkering with the learning rate and a few other usual things that could determine the cause of such NaN value in a dl model but it seems to be of no use. Any idea as to why this might be the case- is the code still been checked for bugs or am I missing something? Thanks in advance!

    opened by SouvikBan 2
Releases(v0.1.1)
Owner
Oleg Smirnov
Oleg Smirnov
Deep learning image registration library for PyTorch

TorchIR: Pytorch Image Registration TorchIR is a image registration library for deep learning image registration (DLIR). I have integrated several ide

Bob de Vos 40 Dec 16, 2022
Code for paper 'Hand-Object Contact Consistency Reasoning for Human Grasps Generation' at ICCV 2021

GraspTTA Hand-Object Contact Consistency Reasoning for Human Grasps Generation (ICCV 2021). Project Page with Videos Demo Quick Results Visualization

Hanwen Jiang 47 Dec 09, 2022
WORD: Revisiting Organs Segmentation in the Whole Abdominal Region

WORD: Revisiting Organs Segmentation in the Whole Abdominal Region. This repository provides the codebase and dataset for our work WORD: Revisiting Or

Healthcare Intelligence Laboratory 71 Jan 07, 2023
Official implementation of ACMMM'20 paper 'Self-supervised Video Representation Learning Using Inter-intra Contrastive Framework'

Self-supervised Video Representation Learning Using Inter-intra Contrastive Framework Official code for paper, Self-supervised Video Representation Le

Li Tao 103 Dec 21, 2022
Unsupervised Video Interpolation using Cycle Consistency

Unsupervised Video Interpolation using Cycle Consistency Project | Paper | YouTube Unsupervised Video Interpolation using Cycle Consistency Fitsum A.

NVIDIA Corporation 100 Nov 30, 2022
🐥A PyTorch implementation of OpenAI's finetuned transformer language model with a script to import the weights pre-trained by OpenAI

PyTorch implementation of OpenAI's Finetuned Transformer Language Model This is a PyTorch implementation of the TensorFlow code provided with OpenAI's

Hugging Face 1.4k Jan 05, 2023
This repository contains the code for "SBEVNet: End-to-End Deep Stereo Layout Estimation" paper by Divam Gupta, Wei Pu, Trenton Tabor, Jeff Schneider

SBEVNet: End-to-End Deep Stereo Layout Estimation This repository contains the code for "SBEVNet: End-to-End Deep Stereo Layout Estimation" paper by D

Divam Gupta 19 Dec 17, 2022
NHL 94 AI contests

nhl94-ai The end goals of this project is to: Train Models that play NHL 94 Support AI vs AI contests in NHL 94 Provide an improved AI opponent for NH

Mathieu Poliquin 2 Dec 06, 2021
Attempt at implementation of a simple GAN using Keras

Simple GAN This is my attempt to make a wrapper class for a GAN in keras which can be used to abstract the whole architecture process. Simple GAN Over

Deven96 7 May 23, 2019
A PyTorch implementation of "CoAtNet: Marrying Convolution and Attention for All Data Sizes".

CoAtNet Overview This is a PyTorch implementation of CoAtNet specified in "CoAtNet: Marrying Convolution and Attention for All Data Sizes", arXiv 2021

Justin Wu 268 Jan 07, 2023
code for CVPR paper Zero-shot Instance Segmentation

Code for CVPR2021 paper Zero-shot Instance Segmentation Code requirements python: python3.7 nvidia GPU pytorch1.1.0 GCC =5.4 NCCL 2 the other python

zhengye 86 Dec 13, 2022
Fully convolutional networks for semantic segmentation

FCN-semantic-segmentation Simple end-to-end semantic segmentation using fully convolutional networks [1]. Takes a pretrained 34-layer ResNet [2], remo

Kai Arulkumaran 186 Dec 25, 2022
PyTorch implementation of "Optimization Planning for 3D ConvNets"

Optimization-Planning-for-3D-ConvNets Code for the ICML 2021 paper: Optimization Planning for 3D ConvNets. Authors: Zhaofan Qiu, Ting Yao, Chong-Wah N

Zhaofan Qiu 2 Jan 12, 2022
Code for "Training Neural Networks with Fixed Sparse Masks" (NeurIPS 2021).

Fisher Induced Sparse uncHanging (FISH) Mask This repo contains the code for Fisher Induced Sparse uncHanging (FISH) Mask training, from "Training Neu

Varun Nair 37 Dec 30, 2022
Addition of pseudotorsion caclulation eta, theta, eta', and theta' to barnaba package

Addition to Original Barnaba Code: This is modified version of Barnaba package to calculate RNA pseudotorsion angles eta, theta, eta', and theta'. Ple

Mandar Kulkarni 1 Jan 11, 2022
Repo for "Benchmarking Robustness of 3D Point Cloud Recognition against Common Corruptions" https://arxiv.org/abs/2201.12296

Benchmarking Robustness of 3D Point Cloud Recognition against Common Corruptions This repo contains the dataset and code for the paper Benchmarking Ro

Jiachen Sun 168 Dec 29, 2022
Supplementary materials for ISMIR 2021 LBD paper "Evaluation of Latent Space Disentanglement in the Presence of Interdependent Attributes"

Evaluation of Latent Space Disentanglement in the Presence of Interdependent Attributes Supplementary materials for ISMIR 2021 LBD submission: K. N. W

Karn Watcharasupat 2 Oct 25, 2021
A crash course in six episodes for software developers who want to become machine learning practitioners.

Featured code sample tensorflow-planespotting Code from the Google Cloud NEXT 2018 session "Tensorflow, deep learning and modern convnets, without a P

Google Cloud Platform 2.6k Jan 08, 2023
A hue shift helper for OBS

obs-hue-shift A hue shift helper for OBS This is a repo based on the really nice script Hegemege made. The original script can be found https://gist.g

Alexis Tyler 1 Jan 10, 2022
Numerical Methods with Python, Numpy and Matplotlib

Numerical Bric-a-Brac Collections of numerical techniques with Python and standard computational packages (Numpy, SciPy, Numba, Matplotlib ...). Diffe

Vincent Bonnet 10 Dec 20, 2021