A library for optimization on Riemannian manifolds

Overview

TensorFlow RiemOpt

PyPI arXiv Build Status Coverage Status Code style: black License

A library for manifold-constrained optimization in TensorFlow.

Installation

To install the latest development version from GitHub:

pip install git+https://github.com/master/tensorflow-riemopt.git

To install a package from PyPI:

pip install tensorflow-riemopt

Features

The core package implements concepts in differential geometry, such as manifolds and Riemannian metrics with associated exponential and logarithmic maps, geodesics, retractions, and transports. For manifolds, where closed-form expressions are not available, the library provides numerical approximations.

import tensorflow_riemopt as riemopt

S = riemopt.manifolds.Sphere()

x = S.projx(tf.constant([0.1, -0.1, 0.1]))
u = S.proju(x, tf.constant([1., 1., 1.]))
v = S.proju(x, tf.constant([-0.7, -1.4, 1.4]))

y = S.exp(x, v)

u_ = S.transp(x, y, u)
v_ = S.transp(x, y, v)

Manifolds

  • manifolds.Cholesky - manifold of lower triangular matrices with positive diagonal elements
  • manifolds.Euclidian - unconstrained manifold with the Euclidean metric
  • manifolds.Grassmannian - manifold of p-dimensional linear subspaces of the n-dimensional space
  • manifolds.Hyperboloid - manifold of n-dimensional hyperbolic space embedded in the n+1-dimensional Minkowski space
  • manifolds.Poincare - the Poincaré ball model of the hyperbolic space
  • manifolds.Product - Cartesian product of manifolds
  • manifolds.SPDAffineInvariant - manifold of symmetric positive definite (SPD) matrices endowed with the affine-invariant metric
  • manifolds.SPDLogCholesky - SPD manifold with the Log-Cholesky metric
  • manifolds.SPDLogEuclidean - SPD manifold with the Log-Euclidean metric
  • manifolds.SpecialOrthogonal - manifold of rotation matrices
  • manifolds.Sphere - manifold of unit-normalized points
  • manifolds.StiefelEuclidean - manifold of orthonormal p-frames in the n-dimensional space endowed with the Euclidean metric
  • manifolds.StiefelCanonical - Stiefel manifold with the canonical metric
  • manifolds.StiefelCayley - Stiefel manifold the retraction map via an iterative Cayley transform

Optimizers

Constrained optimization algorithms work as drop-in replacements for Keras optimizers for sparse and dense updates in both Eager and Graph modes.

  • optimizers.RiemannianSGD - Riemannian Gradient Descent
  • optimizers.RiemannianAdam - Riemannian Adam and AMSGrad
  • optimizers.ConstrainedRMSProp - Constrained RMSProp

Layers

  • layers.ManifoldEmbedding - constrained keras.layers.Embedding layer

Examples

  • SPDNet - Huang, Zhiwu, and Luc Van Gool. "A Riemannian network for SPD matrix learning." Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. AAAI Press, 2017.
  • LieNet - Huang, Zhiwu, et al. "Deep learning on Lie groups for skeleton-based action recognition." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
  • GrNet - Huang, Zhiwu, Jiqing Wu, and Luc Van Gool. "Building Deep Networks on Grassmann Manifolds." AAAI. AAAI Press, 2018.
  • Hyperbolic Neural Network - Ganea, Octavian, Gary Bécigneul, and Thomas Hofmann. "Hyperbolic neural networks." Advances in neural information processing systems. 2018.
  • Poincaré GloVe - Tifrea, Alexandru, Gary Becigneul, and Octavian-Eugen Ganea. "Poincaré Glove: Hyperbolic Word Embeddings." International Conference on Learning Representations. 2018.

References

If you find TensorFlow RiemOpt useful in your research, please cite:

@misc{smirnov2021tensorflow,
      title={TensorFlow RiemOpt: a library for optimization on Riemannian manifolds},
      author={Oleg Smirnov},
      year={2021},
      eprint={2105.13921},
      archivePrefix={arXiv},
      primaryClass={cs.MS}
}

Acknowledgment

TensorFlow RiemOpt was inspired by many similar projects:

  • Manopt, a matlab toolbox for optimization on manifolds
  • Pymanopt, a Python toolbox for optimization on manifolds
  • Geoopt: Riemannian Optimization in PyTorch
  • Geomstats, an open-source Python package for computations and statistics on nonlinear manifolds

License

The code is MIT-licensed.

You might also like...
Distributed Asynchronous Hyperparameter Optimization better than HyperOpt.
Distributed Asynchronous Hyperparameter Optimization better than HyperOpt.

UltraOpt : Distributed Asynchronous Hyperparameter Optimization better than HyperOpt. UltraOpt is a simple and efficient library to minimize expensive

Official code for paper "Optimization for Oriented Object Detection via Representation Invariance Loss".

Optimization for Oriented Object Detection via Representation Invariance Loss By Qi Ming, Zhiqiang Zhou, Lingjuan Miao, Xue Yang, and Yunpeng Dong. Th

Keras + Hyperopt: A very simple wrapper for convenient hyperparameter optimization

This project is now archived. It's been fun working on it, but it's time for me to move on. Thank you for all the support and feedback over the last c

Bayesian optimization in PyTorch

BoTorch is a library for Bayesian Optimization built on PyTorch. BoTorch is currently in beta and under active development! Why BoTorch ? BoTorch Prov

optimization routines for hyperparameter tuning
optimization routines for hyperparameter tuning

Optunity is a library containing various optimizers for hyperparameter tuning. Hyperparameter tuning is a recurrent problem in many machine learning t

Distributed Asynchronous Hyperparameter Optimization in Python

Hyperopt: Distributed Hyperparameter Optimization Hyperopt is a Python library for serial and parallel optimization over awkward search spaces, which

Hyper-parameter optimization for sklearn

hyperopt-sklearn Hyperopt-sklearn is Hyperopt-based model selection among machine learning algorithms in scikit-learn. See how to use hyperopt-sklearn

A Python implementation of global optimization with gaussian processes.
A Python implementation of global optimization with gaussian processes.

Bayesian Optimization Pure Python implementation of bayesian global optimization with gaussian processes. PyPI (pip): $ pip install bayesian-optimizat

Safe Bayesian Optimization
Safe Bayesian Optimization

SafeOpt - Safe Bayesian Optimization This code implements an adapted version of the safe, Bayesian optimization algorithm, SafeOpt [1], [2]. It also p

Comments
  • Projection on SPDs is not projecting onto SPDs

    Projection on SPDs is not projecting onto SPDs

    Hi, nice to see another package doing optimizationon manifolds! I have not yet had the time to check this versus what pymanopt is doing (I think they use tensor flow as a backend, too?) But I just noticed that

    https://github.com/master/tensorflow-manopt/blob/93402f6770d5b3c45f232340fddfa92a7126f19a/tensorflow_manopt/manifolds/symmetric_positive.py#L37-L41

    This might be wrong. For SPDs, the characteristic property is, that all eigenvalues are positive, so this projection is not projection onto the manifold (of SPDs) but onto the set of positive semidefinite matrices. There is no projection onto the SPDs since that set is open in the set of (symmetric) matrices.

    opened by kellertuer 2
  • GrNet produces NaN entries in input tensor

    GrNet produces NaN entries in input tensor

    Hi! First of all, really appreciate you guys taking the time to build a much required riemmannian geometry based package in tensorflow. It is proving to be quite useful for me. However, I recently ran the [GrNet code] (https://github.com/master/tensorflow-riemopt/tree/master/examples/grnet) with the AFEW dataset(the default dataset used in the code) on my machine and it seems at some point the input tensors get filled with NaN values. I tried tinkering with the learning rate and a few other usual things that could determine the cause of such NaN value in a dl model but it seems to be of no use. Any idea as to why this might be the case- is the code still been checked for bugs or am I missing something? Thanks in advance!

    opened by SouvikBan 2
Releases(v0.1.1)
Owner
Oleg Smirnov
Oleg Smirnov
[ICCV 2021] Relaxed Transformer Decoders for Direct Action Proposal Generation

RTD-Net (ICCV 2021) This repo holds the codes of paper: "Relaxed Transformer Decoders for Direct Action Proposal Generation", accepted in ICCV 2021. N

Multimedia Computing Group, Nanjing University 80 Nov 30, 2022
PyTorch implementation of Pay Attention to MLPs

gMLP PyTorch implementation of Pay Attention to MLPs. Quickstart Clone this repository. git clone https://github.com/jaketae/g-mlp.git Navigate to th

Jake Tae 34 Dec 13, 2022
I explore rock vs. mine prediction using a SONAR dataset

I explore rock vs. mine prediction using a SONAR dataset. Using a Logistic Regression Model for my prediction algorithm, I intend on predicting what an object is based on supervised learning.

Jeff Shen 1 Jan 11, 2022
[AAAI-2021] Visual Boundary Knowledge Translation for Foreground Segmentation

Trans-Net Code for (Visual Boundary Knowledge Translation for Foreground Segmentation, AAAI2021). [https://ojs.aaai.org/index.php/AAAI/article/view/16

ZJU-VIPA 2 Mar 04, 2022
Efficient-GlobalPointer - Pytorch Efficient GlobalPointer

引言 感谢苏神带来的模型,原文地址:https://spaces.ac.cn/archives/8877 如何运行 对应模型EfficientGlobalPoi

powerycy 40 Dec 14, 2022
TensorFlow-based neural network library

Sonnet Documentation | Examples Sonnet is a library built on top of TensorFlow 2 designed to provide simple, composable abstractions for machine learn

DeepMind 9.5k Jan 07, 2023
[CVPR 2020] GAN Compression: Efficient Architectures for Interactive Conditional GANs

GAN Compression project | paper | videos | slides [NEW!] GAN Compression is accepted by T-PAMI! We released our T-PAMI version in the arXiv v4! [NEW!]

MIT HAN Lab 1k Jan 07, 2023
A curated list of programmatic weak supervision papers and resources

A curated list of programmatic weak supervision papers and resources

Jieyu Zhang 118 Jan 02, 2023
A convolutional recurrent neural network for classifying A/B phases in EEG signals recorded for sleep analysis.

CAP-Classification-CRNN A deep learning model based on Inception modules paired with gated recurrent units (GRU) for the classification of CAP phases

Apurva R. Umredkar 2 Nov 25, 2022
Includes PyTorch -> Keras model porting code for ConvNeXt family of models with fine-tuning and inference notebooks.

ConvNeXt-TF This repository provides TensorFlow / Keras implementations of different ConvNeXt [1] variants. It also provides the TensorFlow / Keras mo

Sayak Paul 87 Dec 06, 2022
SelfRemaster: SSL Speech Restoration

SelfRemaster: Self-Supervised Speech Restoration Official implementation of SelfRemaster: Self-Supervised Speech Restoration with Analysis-by-Synthesi

Takaaki Saeki 46 Jan 07, 2023
PyTorch implementation for COMPLETER: Incomplete Multi-view Clustering via Contrastive Prediction (CVPR 2021)

Completer: Incomplete Multi-view Clustering via Contrastive Prediction This repo contains the code and data of the following paper accepted by CVPR 20

XLearning Group 72 Dec 07, 2022
Deep Distributed Control of Port-Hamiltonian Systems

De(e)pendable Distributed Control of Port-Hamiltonian Systems (DeepDisCoPH) This repository is associated to the paper [1] and it contains: The full p

Dependable Control and Decision group - EPFL 3 Aug 17, 2022
RCT-ART is an NLP pipeline built with spaCy for converting clinical trial result sentences into tables through jointly extracting intervention, outcome and outcome measure entities and their relations.

Randomised controlled trial abstract result tabulator RCT-ART is an NLP pipeline built with spaCy for converting clinical trial result sentences into

2 Sep 16, 2022
This is the official repository of XVFI (eXtreme Video Frame Interpolation)

XVFI This is the official repository of XVFI (eXtreme Video Frame Interpolation), https://arxiv.org/abs/2103.16206 Last Update: 20210607 We provide th

Jihyong Oh 195 Dec 29, 2022
Retinal Vessel Segmentation with Pixel-wise Adaptive Filters (ISBI 2022)

Retinal Vessel Segmentation with Pixel-wise Adaptive Filters (ISBI 2022) Introdu

anonymous 14 Oct 27, 2022
PEPit is a package enabling computer-assisted worst-case analyses of first-order optimization methods.

PEPit: Performance Estimation in Python This open source Python library provides a generic way to use PEP framework in Python. Performance estimation

Baptiste 53 Nov 16, 2022
Python Implementation of algorithms in Graph Mining, e.g., Recommendation, Collaborative Filtering, Community Detection, Spectral Clustering, Modularity Maximization, co-authorship networks.

Graph Mining Author: Jiayi Chen Time: April 2021 Implemented Algorithms: Network: Scrabing Data, Network Construbtion and Network Measurement (e.g., P

Jiayi Chen 3 Mar 03, 2022
The Official Implementation of Neural View Synthesis and Matching for Semi-Supervised Few-Shot Learning of 3D Pose [NIPS 2021].

Neural View Synthesis and Matching for Semi-Supervised Few-Shot Learning of 3D Pose Release Notes The offical PyTorch implementation of Neural View Sy

Angtian Wang 20 Oct 09, 2022
A Repository of Community-Driven Natural Instructions

A Repository of Community-Driven Natural Instructions TLDR; this repository maintains a community effort to create a large collection of tasks and the

AI2 244 Jan 04, 2023