A library for optimization on Riemannian manifolds

Overview

TensorFlow RiemOpt

PyPI arXiv Build Status Coverage Status Code style: black License

A library for manifold-constrained optimization in TensorFlow.

Installation

To install the latest development version from GitHub:

pip install git+https://github.com/master/tensorflow-riemopt.git

To install a package from PyPI:

pip install tensorflow-riemopt

Features

The core package implements concepts in differential geometry, such as manifolds and Riemannian metrics with associated exponential and logarithmic maps, geodesics, retractions, and transports. For manifolds, where closed-form expressions are not available, the library provides numerical approximations.

import tensorflow_riemopt as riemopt

S = riemopt.manifolds.Sphere()

x = S.projx(tf.constant([0.1, -0.1, 0.1]))
u = S.proju(x, tf.constant([1., 1., 1.]))
v = S.proju(x, tf.constant([-0.7, -1.4, 1.4]))

y = S.exp(x, v)

u_ = S.transp(x, y, u)
v_ = S.transp(x, y, v)

Manifolds

  • manifolds.Cholesky - manifold of lower triangular matrices with positive diagonal elements
  • manifolds.Euclidian - unconstrained manifold with the Euclidean metric
  • manifolds.Grassmannian - manifold of p-dimensional linear subspaces of the n-dimensional space
  • manifolds.Hyperboloid - manifold of n-dimensional hyperbolic space embedded in the n+1-dimensional Minkowski space
  • manifolds.Poincare - the Poincaré ball model of the hyperbolic space
  • manifolds.Product - Cartesian product of manifolds
  • manifolds.SPDAffineInvariant - manifold of symmetric positive definite (SPD) matrices endowed with the affine-invariant metric
  • manifolds.SPDLogCholesky - SPD manifold with the Log-Cholesky metric
  • manifolds.SPDLogEuclidean - SPD manifold with the Log-Euclidean metric
  • manifolds.SpecialOrthogonal - manifold of rotation matrices
  • manifolds.Sphere - manifold of unit-normalized points
  • manifolds.StiefelEuclidean - manifold of orthonormal p-frames in the n-dimensional space endowed with the Euclidean metric
  • manifolds.StiefelCanonical - Stiefel manifold with the canonical metric
  • manifolds.StiefelCayley - Stiefel manifold the retraction map via an iterative Cayley transform

Optimizers

Constrained optimization algorithms work as drop-in replacements for Keras optimizers for sparse and dense updates in both Eager and Graph modes.

  • optimizers.RiemannianSGD - Riemannian Gradient Descent
  • optimizers.RiemannianAdam - Riemannian Adam and AMSGrad
  • optimizers.ConstrainedRMSProp - Constrained RMSProp

Layers

  • layers.ManifoldEmbedding - constrained keras.layers.Embedding layer

Examples

  • SPDNet - Huang, Zhiwu, and Luc Van Gool. "A Riemannian network for SPD matrix learning." Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. AAAI Press, 2017.
  • LieNet - Huang, Zhiwu, et al. "Deep learning on Lie groups for skeleton-based action recognition." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
  • GrNet - Huang, Zhiwu, Jiqing Wu, and Luc Van Gool. "Building Deep Networks on Grassmann Manifolds." AAAI. AAAI Press, 2018.
  • Hyperbolic Neural Network - Ganea, Octavian, Gary Bécigneul, and Thomas Hofmann. "Hyperbolic neural networks." Advances in neural information processing systems. 2018.
  • Poincaré GloVe - Tifrea, Alexandru, Gary Becigneul, and Octavian-Eugen Ganea. "Poincaré Glove: Hyperbolic Word Embeddings." International Conference on Learning Representations. 2018.

References

If you find TensorFlow RiemOpt useful in your research, please cite:

@misc{smirnov2021tensorflow,
      title={TensorFlow RiemOpt: a library for optimization on Riemannian manifolds},
      author={Oleg Smirnov},
      year={2021},
      eprint={2105.13921},
      archivePrefix={arXiv},
      primaryClass={cs.MS}
}

Acknowledgment

TensorFlow RiemOpt was inspired by many similar projects:

  • Manopt, a matlab toolbox for optimization on manifolds
  • Pymanopt, a Python toolbox for optimization on manifolds
  • Geoopt: Riemannian Optimization in PyTorch
  • Geomstats, an open-source Python package for computations and statistics on nonlinear manifolds

License

The code is MIT-licensed.

You might also like...
Distributed Asynchronous Hyperparameter Optimization better than HyperOpt.
Distributed Asynchronous Hyperparameter Optimization better than HyperOpt.

UltraOpt : Distributed Asynchronous Hyperparameter Optimization better than HyperOpt. UltraOpt is a simple and efficient library to minimize expensive

Official code for paper "Optimization for Oriented Object Detection via Representation Invariance Loss".

Optimization for Oriented Object Detection via Representation Invariance Loss By Qi Ming, Zhiqiang Zhou, Lingjuan Miao, Xue Yang, and Yunpeng Dong. Th

Keras + Hyperopt: A very simple wrapper for convenient hyperparameter optimization

This project is now archived. It's been fun working on it, but it's time for me to move on. Thank you for all the support and feedback over the last c

Bayesian optimization in PyTorch

BoTorch is a library for Bayesian Optimization built on PyTorch. BoTorch is currently in beta and under active development! Why BoTorch ? BoTorch Prov

optimization routines for hyperparameter tuning
optimization routines for hyperparameter tuning

Optunity is a library containing various optimizers for hyperparameter tuning. Hyperparameter tuning is a recurrent problem in many machine learning t

Distributed Asynchronous Hyperparameter Optimization in Python

Hyperopt: Distributed Hyperparameter Optimization Hyperopt is a Python library for serial and parallel optimization over awkward search spaces, which

Hyper-parameter optimization for sklearn

hyperopt-sklearn Hyperopt-sklearn is Hyperopt-based model selection among machine learning algorithms in scikit-learn. See how to use hyperopt-sklearn

A Python implementation of global optimization with gaussian processes.
A Python implementation of global optimization with gaussian processes.

Bayesian Optimization Pure Python implementation of bayesian global optimization with gaussian processes. PyPI (pip): $ pip install bayesian-optimizat

Safe Bayesian Optimization
Safe Bayesian Optimization

SafeOpt - Safe Bayesian Optimization This code implements an adapted version of the safe, Bayesian optimization algorithm, SafeOpt [1], [2]. It also p

Comments
  • Projection on SPDs is not projecting onto SPDs

    Projection on SPDs is not projecting onto SPDs

    Hi, nice to see another package doing optimizationon manifolds! I have not yet had the time to check this versus what pymanopt is doing (I think they use tensor flow as a backend, too?) But I just noticed that

    https://github.com/master/tensorflow-manopt/blob/93402f6770d5b3c45f232340fddfa92a7126f19a/tensorflow_manopt/manifolds/symmetric_positive.py#L37-L41

    This might be wrong. For SPDs, the characteristic property is, that all eigenvalues are positive, so this projection is not projection onto the manifold (of SPDs) but onto the set of positive semidefinite matrices. There is no projection onto the SPDs since that set is open in the set of (symmetric) matrices.

    opened by kellertuer 2
  • GrNet produces NaN entries in input tensor

    GrNet produces NaN entries in input tensor

    Hi! First of all, really appreciate you guys taking the time to build a much required riemmannian geometry based package in tensorflow. It is proving to be quite useful for me. However, I recently ran the [GrNet code] (https://github.com/master/tensorflow-riemopt/tree/master/examples/grnet) with the AFEW dataset(the default dataset used in the code) on my machine and it seems at some point the input tensors get filled with NaN values. I tried tinkering with the learning rate and a few other usual things that could determine the cause of such NaN value in a dl model but it seems to be of no use. Any idea as to why this might be the case- is the code still been checked for bugs or am I missing something? Thanks in advance!

    opened by SouvikBan 2
Releases(v0.1.1)
Owner
Oleg Smirnov
Oleg Smirnov
Python implementation of "Single Image Haze Removal Using Dark Channel Prior"

##Dependencies pillow(~2.6.0) Numpy(~1.9.0) If the scripts throw AttributeError: __float__, make sure your pillow has jpeg support e.g. try: $ sudo ap

Joyee Cheung 73 Dec 20, 2022
This repo is duplication of jwyang/faster-rcnn.pytorch

Faster RCNN Pytorch This repo is duplication of jwyang/faster-rcnn.pytorch C/C++ code are removed and easier to study. Python 3.8.5 Ubuntu 20.04.1 LTS

Kim Jihwan 1 Jan 14, 2022
PyTorch ,ONNX and TensorRT implementation of YOLOv4

PyTorch ,ONNX and TensorRT implementation of YOLOv4

4.2k Jan 01, 2023
An efficient PyTorch implementation of the winning entry of the 2017 VQA Challenge.

Bottom-Up and Top-Down Attention for Visual Question Answering An efficient PyTorch implementation of the winning entry of the 2017 VQA Challenge. The

Hengyuan Hu 731 Jan 03, 2023
This MVP data web app uses the Streamlit framework and Facebook's Prophet forecasting package to generate a dynamic forecast from your own data.

📈 Automated Time Series Forecasting Background: This MVP data web app uses the Streamlit framework and Facebook's Prophet forecasting package to gene

Zach Renwick 42 Jan 04, 2023
PRIN/SPRIN: On Extracting Point-wise Rotation Invariant Features

PRIN/SPRIN: On Extracting Point-wise Rotation Invariant Features Overview This repository is the Pytorch implementation of PRIN/SPRIN: On Extracting P

Yang You 17 Mar 02, 2022
Official Pytorch implementation for video neural representation (NeRV)

NeRV: Neural Representations for Videos (NeurIPS 2021) Project Page | Paper | UVG Data Hao Chen, Bo He, Hanyu Wang, Yixuan Ren, Ser-Nam Lim, Abhinav S

hao 214 Dec 28, 2022
DropNAS: Grouped Operation Dropout for Differentiable Architecture Search

DropNAS: Grouped Operation Dropout for Differentiable Architecture Search DropNAS, a grouped operation dropout method for one-level DARTS, with better

weijunhong 4 Aug 15, 2022
Code Release for ICCV 2021 (oral), "AdaFit: Rethinking Learning-based Normal Estimation on Point Clouds"

AdaFit: Rethinking Learning-based Normal Estimation on Point Clouds (ICCV 2021 oral) **Project Page | Arxiv ** Runsong Zhu¹, Yuan Liu², Zhen Dong¹, Te

40 Dec 30, 2022
Official repository for MixFaceNets: Extremely Efficient Face Recognition Networks

MixFaceNets This is the official repository of the paper: MixFaceNets: Extremely Efficient Face Recognition Networks. (Accepted in IJCB2021) https://i

Fadi Boutros 51 Dec 13, 2022
Zeyuan Chen, Yangchao Wang, Yang Yang and Dong Liu.

Principled S2R Dehazing This repository contains the official implementation for PSD Framework introduced in the following paper: PSD: Principled Synt

zychen 78 Dec 30, 2022
Code for ACL 2019 Paper: "COMET: Commonsense Transformers for Automatic Knowledge Graph Construction"

To run a generation experiment (either conceptnet or atomic), follow these instructions: First Steps First clone, the repo: git clone https://github.c

Antoine Bosselut 575 Jan 01, 2023
Speech Recognition is an important feature in several applications used such as home automation, artificial intelligence

Speech Recognition is an important feature in several applications used such as home automation, artificial intelligence, etc. This article aims to provide an introduction on how to make use of the S

RISHABH MISHRA 1 Feb 13, 2022
Definition of a business problem according to Wilson Lower Bound Score and Time Based Average Rating

Wilson Lower Bound Score, Time Based Rating Average In this study I tried to calculate the product rating and sorting reviews more accurately. I have

3 Sep 30, 2021
CIFAR-10 Photo Classification

Image-Classification CIFAR-10 Photo Classification CIFAR-10_Dataset_Classfication CIFAR-10 Photo Classification Dataset CIFAR is an acronym that stand

ADITYA SHAH 1 Jan 05, 2022
Simple reimplemetation experiments about FcaNet

FcaNet-CIFAR An implementation of the paper FcaNet: Frequency Channel Attention Networks on CIFAR10/CIFAR100 dataset. how to run Code: python Cifar.py

76 Feb 04, 2021
An end-to-end machine learning library to directly optimize AUC loss

LibAUC An end-to-end machine learning library for AUC optimization. Why LibAUC? Deep AUC Maximization (DAM) is a paradigm for learning a deep neural n

Andrew 75 Dec 12, 2022
Deepparse is a state-of-the-art library for parsing multinational street addresses using deep learning

Here is deepparse. Deepparse is a state-of-the-art library for parsing multinational street addresses using deep learning. Use deepparse to Use the pr

GRAAL/GRAIL 192 Dec 20, 2022
Models, datasets and tools for Facial keypoints detection

Template for Data Science Project This repo aims to give a robust starting point to any Data Science related project. It contains readymade tools setu

girafe.ai 1 Feb 11, 2022
BiSeNet based on pytorch

BiSeNet BiSeNet based on pytorch 0.4.1 and python 3.6 Dataset Download CamVid dataset from Google Drive or Baidu Yun(6xw4). Pretrained model Download

367 Dec 26, 2022