Implemenets the Contourlet-CNN as described in C-CNN: Contourlet Convolutional Neural Networks, using PyTorch

Overview

C-CNN: Contourlet Convolutional Neural Networks

This repo implemenets the Contourlet-CNN as described in C-CNN: Contourlet Convolutional Neural Networks, using PyTorch, Numpy and Cython.

For texture classification, spectral analysis is traditionally employed in the frequency domain. Recent studies have shown the potential of convolutional neural networks (CNNs) when dealing with the texture classification task in the spatial domain. This network combines both approaches in different domains for more abundant information and proposed a novel network architecture named contourlet CNN (C-CNN). This network aims to learn sparse and effective feature representations for images. First, the contourlet transform is applied to get the spectral features from an image. Second, the spatial-spectral feature fusion strategy is designed to incorporate the spectral features into CNN architecture. Third, the statistical features are integrated into the network by the statistical feature fusion. Finally, the results are obtained by classifying the fusion features.

Installation

The code is tested in a Conda environment setup. First, install PyTorch, torchvision and the appropriate version of cudatoolkit. The code is tested with torch=1.9.1 and torchvision=0.10.1.

conda install pytorch torchvision cudatoolkit=11.1 -c pytorch -c conda-forge

Next, install the other supporting packages from the requirements.txt provided.

pip install -r requirements.txt

You should be able to run the notebooks provided after the setup is done.

Code and Notebooks

In this repo, two Jupyter notebooks is provided.

  1. 01_Visualize_Contourlet_Transform.ipynb - Visualize the contourlet transform output of a sample image, as described in the paper.

  1. 02_Training_DEMO.ipynb - A minimal example of training a Contourlet-CNN on the CIFAR-10 dataset.

The pycontourlet folder contains a modified version of the pycontourlet package from mazayux. Unlike the original, this version works on Python 3.

The contourlet_cnn.py contains the class definition for the Contourlet-CNN network.

Network Variants

The variants of the Contourlet-CNN model. From left to right, each variant is an incremental version of the previous variant, as such in an abalation study in the original paper.

  • "origin" - The 'origin' splices the elongated decomposed images into its corresponding sizes since the contourlet has elongated supports. No SSF features is concatenated to the features in FC2 layer.
  • "SSFF" - Instead of splicing, the 'SSFF' (spatial–spectral feature fusion) via contourlet directly resize the elongated decomposed images into its corresponding sizes. No SSF features is concatenated to the features in FC2 layer.
  • "SSF" - In addition to 'SSFF', the 'SFF' (statistical feature fusion) that denotes the additional texture features of decomposed images, are concatenated to the features in FC2 layer. The mean and variance of each subbands are chosen as the texture features of decomposed images.

In the original paper, the images are converted to grayscale image before feeding into the network. This implementation supports both grayscale images and images with full RGB channels. By setting the spec_type parameter, For full RGB channels, use "all", while to use grayscale images, use "avg".

Examples:

# Uses all RGB channel for contourlet transform, the output are resized, and the statistical
# features are concatenated to the FC layer. This is the recommended variant.
model = ContourletCNN(input_dim=(3, 224, 224), num_classes=10, variant="SSF", spec_type="all")

# Uses only the grayscale channel for contourlet transform, the output are resized, and the 
# statistical features are concatenated to the FC layer.
model = ContourletCNN(input_dim=(3, 224, 224), num_classes=10, variant="SSF", spec_type="avg")

# Uses all RGB channel for contourlet transform, the output are spliced
model = ContourletCNN(input_dim=(3, 224, 224), num_classes=10, variant="origin", spec_type="all")

# Uses all RGB channel for contourlet transform, the output are resized
model = ContourletCNN(input_dim=(3, 224, 224), num_classes=10, variant="SSSF", spec_type="all")
Owner
Goh Kun Shun (KHUN)
Computer Science Major Specializing in Data Science, MMU, Cyberjaya. Currently working as a machine learning engineer,
Goh Kun Shun (KHUN)
This repository contains the exercises and its solution contained in the book "An Introduction to Statistical Learning" in python.

An-Introduction-to-Statistical-Learning This repository contains the exercises and its solution contained in the book An Introduction to Statistical L

2.1k Jan 02, 2023
RoadMap and preparation material for Machine Learning and Data Science - From beginner to expert.

ML-and-DataScience-preparation This repository has the goal to create a learning and preparation roadMap for Machine Learning Engineers and Data Scien

33 Dec 29, 2022
Neural Fixed-Point Acceleration for Convex Optimization

Licensing The majority of neural-scs is licensed under the CC BY-NC 4.0 License, however, portions of the project are available under separate license

Facebook Research 27 Oct 06, 2022
Auxiliary data to the CHIIR paper Searching to Learn with Instructional Scaffolding

Searching to Learn with Instructional Scaffolding This is the data and analysis code for the paper "Searching to Learn with Instructional Scaffolding"

Arthur Câmara 2 Mar 02, 2022
“英特尔创新大师杯”深度学习挑战赛 赛道3:CCKS2021中文NLP地址相关性任务

基于 bert4keras 的一个baseline 不作任何 数据trick 单模 线上 最高可到 0.7891 # 基础 版 train.py 0.7769 # transformer 各层 cls concat 明神的trick https://xv44586.git

孙永松 7 Dec 28, 2021
Disentangled Face Attribute Editing via Instance-Aware Latent Space Search, accepted by IJCAI 2021.

Instance-Aware Latent-Space Search This is a PyTorch implementation of the following paper: Disentangled Face Attribute Editing via Instance-Aware Lat

67 Dec 21, 2022
A script written in Python that returns a consensus string and profile matrix of a given DNA string(s) in FASTA format.

A script written in Python that returns a consensus string and profile matrix of a given DNA string(s) in FASTA format.

Zain 1 Feb 01, 2022
PyTorchMemTracer - Depict GPU memory footprint during DNN training of PyTorch

A Memory Tracer For PyTorch OOM is a nightmare for PyTorch users. However, most

Jiarui Fang 9 Nov 14, 2022
A Pytorch loader for MVTecAD dataset.

MVTecAD A Pytorch loader for MVTecAD dataset. It strictly follows the code style of common Pytorch datasets, such as torchvision.datasets.CIFAR10. The

Jiyuan 1 Dec 27, 2021
Official Pytorch Implementation of Relational Self-Attention: What's Missing in Attention for Video Understanding

Relational Self-Attention: What's Missing in Attention for Video Understanding This repository is the official implementation of "Relational Self-Atte

mandos 43 Dec 07, 2022
Trajectory Extraction of road users via Traffic Camera

Traffic Monitoring Citation The associated paper for this project will be published here as soon as possible. When using this software, please cite th

Julian Strosahl 14 Dec 17, 2022
Code for CVPR2019 Towards Natural and Accurate Future Motion Prediction of Humans and Animals

Motion prediction with Hierarchical Motion Recurrent Network Introduction This work concerns motion prediction of articulate objects such as human, fi

Shuang Wu 85 Dec 11, 2022
Pervasive Attention: 2D Convolutional Networks for Sequence-to-Sequence Prediction

This is a fork of Fairseq(-py) with implementations of the following models: Pervasive Attention - 2D Convolutional Neural Networks for Sequence-to-Se

Maha 490 Dec 15, 2022
Metric learning algorithms in Python

metric-learn: Metric Learning in Python metric-learn contains efficient Python implementations of several popular supervised and weakly-supervised met

1.3k Dec 28, 2022
KUIELAB-MDX-Net got the 2nd place on the Leaderboard A and the 3rd place on the Leaderboard B in the MDX-Challenge ISMIR 2021

KUIELAB-MDX-Net got the 2nd place on the Leaderboard A and the 3rd place on the Leaderboard B in the MDX-Challenge ISMIR 2021

IELab@ Korea University 74 Dec 28, 2022
Code to reproduce the results in the paper "Tensor Component Analysis for Interpreting the Latent Space of GANs".

Tensor Component Analysis for Interpreting the Latent Space of GANs [ paper | project page ] Code to reproduce the results in the paper "Tensor Compon

James Oldfield 4 Jun 17, 2022
CLEAR algorithm for multi-view data association

CLEAR: Consistent Lifting, Embedding, and Alignment Rectification Algorithm The Matlab, Python, and C++ implementation of the CLEAR algorithm, as desc

MIT Aerospace Controls Laboratory 30 Jan 02, 2023
Open AI's Python library

OpenAI Python Library The OpenAI Python library provides convenient access to the OpenAI API from applications written in the Python language. It incl

Pavan Ananth Sharma 3 Jul 10, 2022
Behavioral "black-box" testing for recommender systems

RecList RecList Free software: MIT license Documentation: https://reclist.readthedocs.io. Overview RecList is an open source library providing behavio

Jacopo Tagliabue 375 Dec 30, 2022
A Python library for Deep Probabilistic Modeling

Abstract DeeProb-kit is a Python library that implements deep probabilistic models such as various kinds of Sum-Product Networks, Normalizing Flows an

DeeProb-org 46 Dec 26, 2022