Bayesian inference for Permuton-induced Chinese Restaurant Process (NeurIPS2021).

Overview

Permuton-induced Chinese Restaurant Process

animationMCMCepinions

Note: Currently only the Matlab version is available, but a Python version will be available soon!

This is a demo code for Bayesian nonparametric relational data analysis based on Permuton-induced Chinese Restaurant Process (NeurIPS, 2021). The key features are listed as follows:

  • Clustering based on rectangular partitioning: For an input matrix, the algorithm probabilistically searches for the row and column order and rectangular partitioning so that similar elements are clustered in each block as much as possible.
  • Infinite model complexity: There is no need to fix the suitable number of rectangle clusters in advance, which is a fundamental principle of Bayesian nonparametric machine learning.
  • Arbitrary rectangular partitioning: It can potentially obtain a posterior distribution on arbitrary rectangular partitioning with any numbers of rectangle blocks.
  • Empirically faster mixing of Markov chain Monte Carlo (MCMC) iterations: The method most closely related to this algorithm is the Baxter Permutation Process (NeurIPS, 2020). Typically, this algorithm seems to be able to mix MCMC faster than the Baxter permutation process empirically.

You will need a basic MATLAB installation with Statistics and Machine Learning Toolbox.

In a nutshell

  1. cd permuton-induced-crp
  2. run

Then, the MCMC evolution will appear like the gif animation at the top of this page. The following two items are particularly noteworthy.

  • Top center: Probabilistic rectangular partitioning of a sample matrix (irmdata\sampledata.mat ).
  • Bottom right: Posterior probability.

Interpretation of analysis results

model

The details of the visualization that will be drawn while running the MCMC iterations require additional explanation of our model. Please refer to the paper for more details. Our model, an extension of the Chinese Restaurant Process (CRP), consists of a generative probabilistic model as shown in the figure above (taken from the original paper). While the standard CRP achieves sequence clustering by the analogy of placing customers (data) on tables (clusters), our model additionally achieves array clustering by giving the random table coordinates on [0,1]x[0,1] drawn from the permuton. By viewing the table coordinates as a geometric representation of a permutation, we can use the permutation-to-rectangulation transformation to obtain a rectangular partition of the matrix.

  • Bottom center: Random coordinates of the CRP tables on [0,1]x[0,1]. The size of each table (circle) reflects the number of customers sitting at that table.
  • Top left: Diagonal rectangulation corresponding to the permutation represented by the table coordinates.
  • Bottom left: Generic rectangulation corresponding to the permutation represented by the table coordinates.

Details of usage

Given an input relational matrix, the Permuton-induced Chinese Restaurant Process can be fitted to it by a MCMC inference algorithm as follows:

[RowTable, ColumnTable, TableCoordinates, nesw] = test_MCMC_PCRP(X);

or

[RowTable, ColumnTable, TableCoordinates, nesw] = test_MCMC_PCRP(X, opt);

  • X: An M by N input observation matrix. Each element must be natural numbers.
  • opt.maxiter: Maximum number of MCMC iterations.
  • opt.missingRatio: Ratio of test/(training+test) for prediction performance evaluation based on perplexity.

Reference

  1. M. Nakano, Yasuhiro Fujiwara, A. Kimura, T. Yamada, and N. Ueda, 'Permuton-induced Chinese Restaurant Process,' Advances in Neural Information Processing Systems 34 (NeurIPS 2021).

    @inproceedings{Nakano2021,
     author = {Nakano, Masahiro and Fujiwara, Yasuhiro and Kimura, Akisato and Yamada, Takeshi and Ueda, Naonori},
     booktitle = {Advances in Neural Information Processing Systems},
     pages = {},
     publisher = {Curran Associates, Inc.},
     title = {Permuton-induced Chinese Restaurant Process},
     url = {},
     volume = {34},
     year = {2021}
    }
    
Owner
NTT Communication Science Laboratories
NTT Communication Science Laboratories
Solving Zero-Shot Learning in Named Entity Recognition with Common Sense Knowledge

Zero-Shot Learning in Named Entity Recognition with Common Sense Knowledge Associated code for the paper Zero-Shot Learning in Named Entity Recognitio

Søren Hougaard Mulvad 13 Dec 25, 2022
Jigsaw Rate Severity of Toxic Comments

Jigsaw Rate Severity of Toxic Comments

Guanshuo Xu 66 Nov 30, 2022
Algorithmic Trading using RNN

Deep-Trading This an implementation adapted from Rachnog Neural networks for algorithmic trading. Part One — Simple time series forecasting and this c

Hazem Nomer 29 Sep 04, 2022
Prototype python implementation of the ome-ngff table spec

Prototype python implementation of the ome-ngff table spec

Kevin Yamauchi 8 Nov 20, 2022
Intro-to-dl - Resources for "Introduction to Deep Learning" course.

Introduction to Deep Learning course resources https://www.coursera.org/learn/intro-to-deep-learning Running on Google Colab (tested for all weeks) Go

Advanced Machine Learning specialisation by HSE 761 Dec 24, 2022
Experimenting with computer vision techniques to generate annotated image datasets from gameplay recordings automatically.

Experimenting with computer vision techniques to generate annotated image datasets from gameplay recordings automatically. The collected data will then be used to train a deep neural network that can

Martin Valchev 3 Apr 24, 2022
Text-to-Music Retrieval using Pre-defined/Data-driven Emotion Embeddings

Text2Music Emotion Embedding Text-to-Music Retrieval using Pre-defined/Data-driven Emotion Embeddings Reference Emotion Embedding Spaces for Matching

Minz Won 50 Dec 05, 2022
GPT, but made only out of gMLPs

GPT - gMLP This repository will attempt to crack long context autoregressive language modeling (GPT) using variations of gMLPs. Specifically, it will

Phil Wang 80 Dec 01, 2022
An unofficial PyTorch implementation of a federated learning algorithm, FedAvg.

Federated Averaging (FedAvg) in PyTorch An unofficial implementation of FederatedAveraging (or FedAvg) algorithm proposed in the paper Communication-E

Seok-Ju Hahn 123 Jan 06, 2023
The code for our paper "AutoSF: Searching Scoring Functions for Knowledge Graph Embedding"

AutoSF The code for our paper "AutoSF: Searching Scoring Functions for Knowledge Graph Embedding" and this paper has been accepted by ICDE2020. News:

AutoML Research 64 Dec 17, 2022
This repo includes our code for evaluating and improving transferability in domain generalization (NeurIPS 2021)

Transferability for domain generalization This repo is for evaluating and improving transferability in domain generalization (NeurIPS 2021), based on

gordon 9 Nov 29, 2022
Analysis of Smiles through reservoir sampling & RDkit

Analysis of Smiles through reservoir sampling and machine learning (under development). This is a simple project that includes two Jupyter files for t

Aurimas A. Nausėdas 6 Aug 30, 2022
This repository gives an example on how to preprocess the data of the HECKTOR challenge

HECKTOR 2021 challenge This repository gives an example on how to preprocess the data of the HECKTOR challenge. Any other preprocessing is welcomed an

56 Dec 01, 2022
Fine-grained Control of Image Caption Generation with Abstract Scene Graphs

Faster R-CNN pretrained on VisualGenome This repository modifies maskrcnn-benchmark for object detection and attribute prediction on VisualGenome data

Shizhe Chen 7 Apr 20, 2021
Event-forecasting - Event Forecasting Algorithms With Python

event-forecasting Event Forecasting Algorithms Theory Correlating events in comp

Intellia ICT 4 Feb 15, 2022
3D Avatar Lip Syncronization from speech (JALI based face-rigging)

visemenet-inference Inference Demo of "VisemeNet-tensorflow" VisemeNet is an audio-driven animator centric speech animation driving a JALI or standard

Junhwan Jang 17 Dec 20, 2022
Expand human face editing via Global Direction of StyleCLIP, especially to maintain similarity during editing.

Oh-My-Face This project is based on StyleCLIP, RIFE, and encoder4editing, which aims to expand human face editing via Global Direction of StyleCLIP, e

AiLin Huang 51 Nov 17, 2022
The Ludii general game system, developed as part of the ERC-funded Digital Ludeme Project.

The Ludii General Game System Ludii is a general game system being developed as part of the ERC-funded Digital Ludeme Project (DLP). This repository h

Digital Ludeme Project 50 Jan 04, 2023
GyroSPD: Vector-valued Distance and Gyrocalculus on the Space of Symmetric Positive Definite Matrices

GyroSPD Code for the paper "Vector-valued Distance and Gyrocalculus on the Space of Symmetric Positive Definite Matrices" accepted at NeurIPS 2021. Re

Federico Lopez 12 Dec 12, 2022
A Pytree Module system for Deep Learning in JAX

Treex A Pytree-based Module system for Deep Learning in JAX Intuitive: Modules are simple Python objects that respect Object-Oriented semantics and sh

Cristian Garcia 216 Dec 20, 2022