Hierarchical Uniform Manifold Approximation and Projection

Overview

pypi_version pypi_downloads

HUMAP exploration on Fashion MNIST dataset

HUMAP

Hierarchical Manifold Approximation and Projection (HUMAP) is a technique based on UMAP for hierarchical non-linear dimensionality reduction. HUMAP allows to:

  1. Focus on important information while reducing the visual burden when exploring whole datasets;
  2. Drill-down the hierarchy according to information demand.

The details of the algorithm can be found in our paper on ArXiv.

Installation

HUMAP was written in C++ for performance purposes, and it has an intuitive Python interface. It depends upon common machine learning libraries, such as scikit-learn and NumPy. It also needs the pybind11 due to the interface between C++ and Python.

Requirements:

  • Python 3.6 or greater
  • numpy
  • scipy
  • scikit-learn
  • pybind11
  • Eigen (C++)

If you have these requirements installed, use PyPI:

pip install humap

For Windows users:

The Eigen library does not have to be installed. Just add the files to C:Eigen or use the manual installation to change Eigen location.

Manual installation:

For manually installing HUMAP, download the project and proceed as follows:

python setup.py bdist_wheel
pip install dist/humap*.whl

Usage examples

HUMAP package follows the same idea of sklearn classes, in which you need to fit and transform data.

Fitting the hierarchy

import humap
from sklearn.datasets import fetch_openml


X, y = fetch_openml('mnist_784', version=1, return_X_y=True)

hUmap = humap.HUMAP()
hUmap.fit(X, y)

HUMAP embedding of top-level MNIST digits

By now, you can control six parameters related to the hierarchy construction and the embedding performed by UMAP.

  • levels: Controls the number of hierarchical levels + the first one (whole dataset). This parameter also controls how many data points are in each hierarchical level. The default is [0.2, 0.2], meaning the HUMAP will produce three levels: The first one with the whole dataset, the second one with 20% of the first level, and the third with 20% of the second level.
  • n_neighbors: This parameter controls the number of neighbors for approximating the manifold structures. Larger values produce embedding that preserves more of the global relations. In HUMAP, we recommend and set the default value to be 100.
  • min_dist: This parameter, used in UMAP dimensionality reduction, controls the allowance to cluster data points together. According to UMAP documentation, larger values allow evenly distributed embeddings, while smaller values encode the local structures better. We set this parameter as 0.15 as default.
  • knn_algorithm: Controls which knn approximation will be used, in which NNDescent is the default. Another option is ANNOY or FLANN if you have Python installations of these algorithms at the expense of slower run-time executions than NNDescent.
  • init: Controls the method for initing the low-dimensional representation. We set Spectral as default since it yields better global structure preservation. You can also use random initialization.
  • verbose: Controls the verbosity of the algorithm.

Embedding a hierarchical level

After fitting the dataset, you can generate the embedding for a hierarchical level by specifying the level.

embedding_l2 = hUmap.transform(2)
y_l2 = hUmap.labels(2)

Notice that the .labels() method only works for levels equal or greater than one.

Drilling down the hierarchy by embedding a subset of data points based on indices

Embedding data subsets throughout HUMAP hierarchy

When interested in a set of data samples, HUMAP allows for drilling down the hierarchy for those samples.

embedding, y, indices = hUmap.transform(2, indices=indices_of_interest)

This method returns the embedding coordinates, the labels (y), and the data points' indices in the current level. Notice that the current level is now level 1 since we used the hierarchy level 2 for drilling down operation.

Drilling down the hierarchy by embedding a subset of data points based on labels

You can apply the same concept as above to embed data points based on labels.

embedding, y, indices = hUmap.transform(2, indices=np.array([4, 9]), class_based=True)

C++ UMAP implementation

You can also fit a one-level HUMAP hierarchy, which essentially corresponds to a UMAP projection.

umap_reducer = humap.HUMAP(np.array([]))
umap_reducer.fit(X, y)

embedding = umap_reducer.transform(0)

Citation

Please, use the following reference to cite HUMAP in your work:

@misc{marciliojr_humap2021,
  title={HUMAP: Hierarchical Uniform Manifold Approximation and Projection},
  author={Wilson E. Marcílio-Jr and Danilo M. Eler and Fernando V. Paulovich and Rafael M. Martins},
  year={2021},
  eprint={2106.07718},
  archivePrefix={arXiv},
  primaryClass={cs.LG}
    }

License

HUMAP follows the 3-clause BSD license and it uses the open-source NNDescent implementation from EFANNA. It also uses a C++ implementation of UMAP for embedding hierarchy levels; this project would not be possible without UMAP's fantastic technique and package.

E-mail me (wilson_jr at outlook.com) if you like to contribute.


You might also like...
Finite-temperature variational Monte Carlo calculation of uniform electron gas using neural canonical transformation.

CoulombGas This code implements the neural canonical transformation approach to the thermodynamic properties of uniform electron gas. Building on JAX,

Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation. In CVPR 2022.
Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation. In CVPR 2022.

Nonuniform-to-Uniform Quantization This repository contains the training code of N2UQ introduced in our CVPR 2022 paper: "Nonuniform-to-Uniform Quanti

Curved Projection Reformation
Curved Projection Reformation

Description Assuming that we already know the image of the centerline, we want the lumen to be displayed on a plane, which requires curved projection

Implementation of
Implementation of "Bidirectional Projection Network for Cross Dimension Scene Understanding" CVPR 2021 (Oral)

Bidirectional Projection Network for Cross Dimension Scene Understanding CVPR 2021 (Oral) [ Project Webpage ] [ arXiv ] [ Video ] Existing segmentatio

my graduation project is about live human face augmentation by projection mapping by using CNN
my graduation project is about live human face augmentation by projection mapping by using CNN

Live-human-face-expression-augmentation-by-projection my graduation project is about live human face augmentation by projection mapping by using CNN o

Official repository for the ICLR 2021 paper Evaluating the Disentanglement of Deep Generative Models with Manifold Topology

Official repository for the ICLR 2021 paper Evaluating the Disentanglement of Deep Generative Models with Manifold Topology Sharon Zhou, Eric Zelikman

Code for Learning Manifold Patch-Based Representations of Man-Made Shapes, in ICLR 2021.
Code for Learning Manifold Patch-Based Representations of Man-Made Shapes, in ICLR 2021.

LearningPatches | Webpage | Paper | Video Learning Manifold Patch-Based Representations of Man-Made Shapes Dmitriy Smirnov, Mikhail Bessmeltsev, Justi

A Pytorch implementation of
A Pytorch implementation of "Manifold Matching via Deep Metric Learning for Generative Modeling" (ICCV 2021)

Manifold Matching via Deep Metric Learning for Generative Modeling A Pytorch implementation of "Manifold Matching via Deep Metric Learning for Generat

Research Artifact of USENIX Security 2022 Paper: Automated Side Channel Analysis of Media Software with Manifold Learning

Manifold-SCA Research Artifact of USENIX Security 2022 Paper: Automated Side Channel Analysis of Media Software with Manifold Learning The repo is org

Comments
  • [Packaging] Requesting conda-forge package

    [Packaging] Requesting conda-forge package

    Hi,

    Just putting it out there that you might want to consider putting up your package on conda-forge. Many other packages like numpy, scikit-learn, umap, are all available on conda-forge, and managing them through conda cli makes it easy to be up-to-date and not worry about dependencies like MKL, which pip doesn't handle well.

    As a bonus, I see that this package depends on Eigen, which needs to be manually configured on Windows. Conda-forge already has eigen available, which might make this much less error-prone for Windows users, which I assume will be a substantial chunk.

    Just as an FYI, here is a link for conda-forge submission process.

    Thanks!

    opened by stallam-unb 6
  • RuntimeError: Some rows contain fewer than n_neighbors distances

    RuntimeError: Some rows contain fewer than n_neighbors distances

    Problems when computing hierarchy for small datasets. I tried to execute HUMAP on Iris dataset using 100, 15, and 10 n_neighbors.

    RuntimeError: Some rows contain fewer than n_neighbors distances

    opened by wilsonjr 1
  • Transform with new data?

    Transform with new data?

    Semi-related to #4 , but my case is that I want to use HUMAP on a supervised data where I have a training data with labels, and I want to be able to project new test data with the same embeddings. UMAP supports this use case, I was wondering if this would be theoretically possible with HUMAP as well? Would be nice to be able to use HUMAP to interpret classifier decisions.

    opened by stallam-unb 0
  • Semi-supervised learning?

    Semi-supervised learning?

    Thanks for writing this awesome library, only recently discovered it. Do you have plans to support semi-supervised umap? From my first try outs of your library, this is the fastest (h)umap implementation which has nndescent. I would like to use it for semi-supervised learning, too.

    enhancement 
    opened by KnutJaegersberg 6
Releases(v0.2.1)
Owner
Wilson Estécio Marcílio Júnior
PhD Candidate in Computer Science. Interested in ML and Explainability.
Wilson Estécio Marcílio Júnior
Multiple-criteria decision-making (MCDM) with Electre, Promethee, Weighted Sum and Pareto

EasyMCDM - Quick Installation methods Install with PyPI Once you have created your Python environment (Python 3.6+) you can simply type: pip3 install

Labrak Yanis 6 Nov 22, 2022
A Dying Light 2 (DL2) PAKFile Utility for Modders and Mod Makers.

Dying Light 2 PAKFile Utility A Dying Light 2 (DL2) PAKFile Utility for Modders and Mod Makers. This tool aims to make PAKFile (.pak files) modding a

RHQ Online 12 Aug 26, 2022
Malware Env for OpenAI Gym

Malware Env for OpenAI Gym Citing If you use this code in a publication please cite the following paper: Hyrum S. Anderson, Anant Kharkar, Bobby Fila

ENDGAME 563 Dec 29, 2022
Pytorch implementation of TailCalibX : Feature Generation for Long-tail Classification

TailCalibX : Feature Generation for Long-tail Classification by Rahul Vigneswaran, Marc T. Law, Vineeth N. Balasubramanian, Makarand Tapaswi [arXiv] [

Rahul Vigneswaran 34 Jan 02, 2023
OpenMMLab Image and Video Editing Toolbox

Introduction MMEditing is an open source image and video editing toolbox based on PyTorch. It is a part of the OpenMMLab project. The master branch wo

OpenMMLab 3.9k Jan 04, 2023
[ICML 2021] Break-It-Fix-It: Learning to Repair Programs from Unlabeled Data

Break-It-Fix-It: Learning to Repair Programs from Unlabeled Data This repo provides the source code & data of our paper: Break-It-Fix-It: Unsupervised

Michihiro Yasunaga 86 Nov 30, 2022
Styleformer - Official Pytorch Implementation

Styleformer -- Official PyTorch implementation Styleformer: Transformer based Generative Adversarial Networks with Style Vector(https://arxiv.org/abs/

Jeeseung Park 159 Dec 12, 2022
AnimationKit: AI Upscaling & Interpolation using Real-ESRGAN+RIFE

ALPHA 2.5: Frostbite Revival (Released 12/23/21) Changelog: [ UI ] Chained design. All steps link to one another! Use the master override toggles to s

87 Nov 16, 2022
Fast image augmentation library and easy to use wrapper around other libraries. Documentation: https://albumentations.ai/docs/ Paper about library: https://www.mdpi.com/2078-2489/11/2/125

Albumentations Albumentations is a Python library for image augmentation. Image augmentation is used in deep learning and computer vision tasks to inc

11.4k Jan 09, 2023
PyJokes - Joking around with Python library pyjokes

Hi, it's Muhaimin again 👋 This is something unorthodox but cool. Don't forget t

Muhaimin A. Salay Kanton 1 Feb 02, 2022
Select, weight and analyze complex sample data

Sample Analytics In large-scale surveys, often complex random mechanisms are used to select samples. Estimates derived from such samples must reflect

samplics 37 Dec 15, 2022
Warning: This project does not have any current developer. See bellow.

Pylearn2: A machine learning research library Warning : This project does not have any current developer. We will continue to review pull requests and

Laboratoire d’Informatique des Systèmes Adaptatifs 2.7k Dec 26, 2022
Official repository for "Restormer: Efficient Transformer for High-Resolution Image Restoration". SOTA for motion deblurring, image deraining, denoising (Gaussian/real data), and defocus deblurring.

Restormer: Efficient Transformer for High-Resolution Image Restoration Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan,

Syed Waqas Zamir 906 Dec 30, 2022
A PyTorch implementation of Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks

SVHNClassifier-PyTorch A PyTorch implementation of Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks If

Potter Hsu 182 Jan 03, 2023
For visualizing the dair-v2x-i dataset

3D Detection & Tracking Viewer The project is based on hailanyi/3D-Detection-Tracking-Viewer and is modified, you can find the original version of the

34 Dec 29, 2022
All of the figures and notebooks for my deep learning book, for free!

"Deep Learning - A Visual Approach" by Andrew Glassner This is the official repo for my book from No Starch Press. Ordering the book My book is called

Andrew Glassner 227 Jan 04, 2023
Implementation of a Transformer using ReLA (Rectified Linear Attention)

ReLA (Rectified Linear Attention) Transformer Implementation of a Transformer using ReLA (Rectified Linear Attention). It will also contain an attempt

Phil Wang 49 Oct 14, 2022
Codebase for Time-series Generative Adversarial Networks (TimeGAN)

Codebase for Time-series Generative Adversarial Networks (TimeGAN)

Jinsung Yoon 532 Dec 31, 2022
PyTorch implementation of the TTC algorithm

Trust-the-Critics This repository is a PyTorch implementation of the TTC algorithm and the WGAN misalignment experiments presented in Trust the Critic

0 Nov 29, 2021
Public repository created to store my custom-made tools for Just Dance (UbiArt Engine)

Woody's Just Dance Tools Public repository created to store my custom-made tools for Just Dance (UbiArt Engine) Development and updates Almost all of

Wodson de Andrade 8 Dec 24, 2022