This repository contains the source code and data for reproducing results of Deep Continuous Clustering paper

Overview

Deep Continuous Clustering

Introduction

This is a Pytorch implementation of the DCC algorithms presented in the following paper (paper):

Sohil Atul Shah and Vladlen Koltun. Deep Continuous Clustering.

If you use this code in your research, please cite our paper.

@article{shah2018DCC,
	author    = {Sohil Atul Shah and Vladlen Koltun},
	title     = {Deep Continuous Clustering},
	journal   = {arXiv:1803.01449},
	year      = {2018},
}

The source code and dataset are published under the MIT license. See LICENSE for details. In general, you can use the code for any purpose with proper attribution. If you do something interesting with the code, we'll be happy to know. Feel free to contact us.

Requirement

Pretraining SDAE

Note: Please find required files and checkpoints for MNIST dataset shared here.

Please create new folder for each dataset under the data folder. Please follow the structure of mnist dataset. The training and the validation data for each dataset must be placed under their respective folder.

We have already provided train and test data files for MNIST dataset. For example, one can start pretraining of SDAE from console as follows:

$ python pretraining.py --data mnist --tensorboard --id 1 --niter 50000 --lr 10 --step 20000

Different settings for total iterations, learning rate and stepsize may be required for other datasets. Please find the details under the comment section inside the pretraining file.

Extracting Pretrained Features

The features from the pretrained SDAE network are extracted as follows:

$ python extract_feature.py --data mnist --net checkpoint_4.pth.tar --features pretrained

By default, the model checkpoint for pretrained SDAE NW is stored under results.

Copying mkNN graph

The copyGraph program is used to merge the preprocessed mkNN graph (using the code provided by RCC) and the extracted pretrained features. Note the mkNN graph is built on the original and not on the SDAE features.

$ python copyGraph.py --data mnist --graph pretrained.mat --features pretrained.pkl --out pretrained

The above command assumes that the graph is stored in the pretrained.mat file and the merged file is stored back to pretrained.mat file.

DCC searches for the file with name pretrained.mat. Hence please retain the name.

Running Deep Continuous Clustering

Once the features are extracted and graph details merged, one can start training DCC algorithm.

For sanity check, we have also provided a pretrained.mat and SDAE model files for the MNIST dataset located under the data folder. For example, one can run DCC on MNIST from console as follows:

$ python DCC.py --data mnist --net checkpoint_4.pth.tar --tensorboard --id 1

The other preprocessed graph files can be found in gdrive folder as provided by the RCC.

Evaluation

Towards the end of run of DCC algorithm, i.e., once the stopping criterion is met, DCC starts evaluating the cluster assignment for the total dataset. The evaluation output is logged into tensorboard logger. The penultimate evaluated output is reported in the paper.

Like RCC, the AMI definition followed here differs slightly from the default definition found in the sklearn package. To match the results listed in the paper, please modify it accordingly.

The tensorboard logs for both pretraining and DCC will be stored in the "runs/DCC" folder under results. The final embedded features 'U' and cluster assignment for each sample is saved in 'features.mat' file under results.

Creating input

The input file for SDAE pretraining, traindata.mat and testdata.mat, stores the features of the 'N' data samples in a matrix format N x D. We followed 4:1 ratio to split train and validation data. The provided make_data.py can be used to build training and validation data. The distinction of training and validation set is used only for the pretraining stage. For end-to-end training, there is no such distinction in unsupervised learning and hence all data has been used.

To construct mkNN edge set and to create preprocessed input file, pretrained.mat, from the raw feature file, use edgeConstruction.py released by RCC. Please follow the instruction therein. Note that mkNN graph is built on the complete dataset. For simplicity, code (post pretraining phase) follows the data ordering of [trainset, testset] to arrange the data. This should be consistent even with mkNN construction.

Understanding Steps Through Visual Example

Generate 2D clustered data with

python make_data.py --data easy

This creates 3 clusters where the centers are colinear to each other. We would then expect to only need 1 dimensional latent space (either x or y) to uniquely project the data onto the line passing through the center of the clusters.

generated ground truth

Construct mKNN graph with

python edgeConstruction.py --dataset easy --samples 600

Pretrain SDAE with

python pretraining.py --data easy --tensorboard --id 1 --niter 500 --dim 1 --lr 0.0001 --step 300

You can debug the pretraining losses using tensorboard (needs tensorflow) with

tensorboard --logdir data/easy/results/runs/pretraining/1/

Then navigate to the http link that is logged in console.

Extract pretrained features

python extract_feature.py --data easy --net checkpoint_2.pth.tar --features pretrained --dim 1

Merge preprocessed mkNN graph and the pretrained features with

python copyGraph.py --data easy --graph pretrained.mat --features pretrained.pkl --out pretrained

Run DCC with

python DCC.py --data easy --net checkpoint_2.pth.tar --tensorboard --id 1 --dim 1

Debug and show how the representatives shift over epochs with

tensorboard --logdir data/easy/results/runs/DCC/1/ --samples_per_plugin images=100

Pretraining and DCC together in one script

See easy_example.py for the previous easy to visualize example all steps done in one script. Execute the script to perform the previous section all together. You can visualize the results, such as how the representatives drift over iterations with the tensorboard command above and navigating to the Images tab.

With an autoencoder, the representatives shift over epochs like: shift with autoencoder

Owner
Sohil Shah
Research Scientist
Sohil Shah
Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers (arXiv2021)

Polyp-PVT by Bo Dong, Wenhai Wang, Deng-Ping Fan, Jinpeng Li, Huazhu Fu, & Ling Shao. This repo is the official implementation of "Polyp-PVT: Polyp Se

Deng-Ping Fan 102 Jan 05, 2023
Inhomogeneous Social Recommendation with Hypergraph Convolutional Networks

Inhomogeneous Social Recommendation with Hypergraph Convolutional Networks This is our Pytorch implementation for the paper: Zirui Zhu, Chen Gao, Xu C

Zirui Zhu 3 Dec 30, 2022
docTR by Mindee (Document Text Recognition) - a seamless, high-performing & accessible library for OCR-related tasks powered by Deep Learning.

docTR by Mindee (Document Text Recognition) - a seamless, high-performing & accessible library for OCR-related tasks powered by Deep Learning.

Mindee 1.5k Jan 01, 2023
This repository is the official implementation of the Hybrid Self-Attention NEAT algorithm.

This repository is the official implementation of the Hybrid Self-Attention NEAT algorithm. It contains the code to reproduce the results presented in the original paper: https://arxiv.org/abs/2112.0

Saman Khamesian 6 Dec 13, 2022
OntoProtein: Protein Pretraining With Ontology Embedding

OntoProtein This is the implement of the paper "OntoProtein: Protein Pretraining With Ontology Embedding". OntoProtein is an effective method that mak

ZJUNLP 80 Dec 14, 2022
SphereFace: Deep Hypersphere Embedding for Face Recognition

SphereFace: Deep Hypersphere Embedding for Face Recognition By Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj and Le Song License SphereFa

Weiyang Liu 1.5k Dec 29, 2022
Core ML tools contain supporting tools for Core ML model conversion, editing, and validation.

Core ML Tools Use coremltools to convert machine learning models from third-party libraries to the Core ML format. The Python package contains the sup

Apple 3k Jan 08, 2023
Decoding the Protein-ligand Interactions Using Parallel Graph Neural Networks

Decoding the Protein-ligand Interactions Using Parallel Graph Neural Networks Requirements python 0.10+ rdkit 2020.03.3.0 biopython 1.78 openbabel 2.4

Neeraj Kumar 3 Nov 23, 2022
This repository is an implementation of paper : Improving the Training of Graph Neural Networks with Consistency Regularization

CRGNN Paper : Improving the Training of Graph Neural Networks with Consistency Regularization Environments Implementing environment: GeForce RTX™ 3090

THUDM 28 Dec 09, 2022
Pytorch implementation of the paper DocEnTr: An End-to-End Document Image Enhancement Transformer.

DocEnTR Description Pytorch implementation of the paper DocEnTr: An End-to-End Document Image Enhancement Transformer. This model is implemented on to

Mohamed Ali Souibgui 74 Jan 07, 2023
An atmospheric growth and evolution model based on the EVo degassing model and FastChem 2.0

EVolve Linking planetary mantles to atmospheric chemistry through volcanism using EVo and FastChem. Overview EVolve is a linked mantle degassing and a

Pip Liggins 2 Jan 17, 2022
This repository contains the code for the paper Neural RGB-D Surface Reconstruction

Neural RGB-D Surface Reconstruction Paper | Project Page | Video Neural RGB-D Surface Reconstruction Dejan Azinović, Ricardo Martin-Brualla, Dan B Gol

Dejan 406 Jan 04, 2023
Hyperbolic Hierarchical Clustering.

Hyperbolic Hierarchical Clustering (HypHC) This code is the official PyTorch implementation of the NeurIPS 2020 paper: From Trees to Continuous Embedd

HazyResearch 154 Dec 15, 2022
A Closer Look at Reference Learning for Fourier Phase Retrieval

A Closer Look at Reference Learning for Fourier Phase Retrieval This repository contains code for our NeurIPS 2021 Workshop on Deep Learning and Inver

Tobias Uelwer 1 Oct 28, 2021
[NeurIPS 2021] Galerkin Transformer: a linear attention without softmax

[NeurIPS 2021] Galerkin Transformer: linear attention without softmax Summary A non-numerical analyst oriented explanation on Toward Data Science abou

Shuhao Cao 159 Dec 20, 2022
Learnable Motion Coherence for Correspondence Pruning

Learnable Motion Coherence for Correspondence Pruning Yuan Liu, Lingjie Liu, Cheng Lin, Zhen Dong, Wenping Wang Project Page Any questions or discussi

liuyuan 41 Nov 30, 2022
Woosung Choi 63 Nov 14, 2022
Spatially-Adaptive Pixelwise Networks for Fast Image Translation, CVPR 2021

Image Translation with ASAPNets Spatially-Adaptive Pixelwise Networks for Fast Image Translation, CVPR 2021 Webpage | Paper | Video Installation insta

Tamar Rott Shaham 100 Dec 28, 2022
African language Speech Recognition - Speech-to-Text

Swahili-Speech-To-Text Table of Contents Swahili-Speech-To-Text Overview Scenario Approach Project Structure data: models: notebooks: scripts tests: l

2 Jan 05, 2023
Ranger deep learning optimizer rewrite to use newest components

Ranger21 - integrating the latest deep learning components into a single optimizer Ranger deep learning optimizer rewrite to use newest components Ran

Less Wright 266 Dec 28, 2022