Code of the paper "Part Detector Discovery in Deep Convolutional Neural Networks" by Marcel Simon, Erik Rodner and Joachim Denzler

Overview

Part Detector Discovery

This is the code used in our paper "Part Detector Discovery in Deep Convolutional Neural Networks" by Marcel Simon, Erik Rodner and Joachim Denzler published at ACCV 2014. If you would like to refer to this work, please cite the corresponding paper

@inproceedings{Simon14:PDD,
  author = {Marcel Simon and Erik Rodner and Joachim Denzler},
  booktitle = {Asian Conference on Computer Vision (ACCV)},
  title = {Part Detector Discovery in Deep Convolutional Neural Networks},
  year = {2014},
}

The following steps will guide you through the usage of the code.

1. Python Environment

Setup a python environment, preferably a virtual environment using e. g. virtual_env. The requirements file might install more than you need.

virtualenv pyhton-env && pip install -r requirements.txt

2. DeCAF Installation

Build and install decaf into this environment

source python-env/bin/activate
cd decaf-tools/decaf/
python setup.py build
python setup.py install

3. Pre-Trained ImageNet Model

Get the decaf ImageNet model:

cd decaf-tools/models/
bash get_model.sh

You now might need to adjust the path to the decaf model in decaf-tools/extract_grad_map.py, line 75!

4. Gradient Map Calculation

Now you can calculate the gradient maps using the following command. For a single image, use decaf-tools/extract_grad_map.py :

usage: extract_grad_map.py [-h] [--layers LAYERS [LAYERS ...]] [--limit LIMIT]
                           [--channel_limit CHANNEL_LIMIT]
                           [--images pattern [pattern ...]] [--outdir OUTDIR]

Calculate the gradient maps for an image.

optional arguments:
  -h, --help            show this help message and exit
  --layers LAYERS [LAYERS ...]
  --limit LIMIT         When calculating the gradient of the class scores,
                        calculate the gradient for the output elements with the
                        [limit] highest probabilities.
  --channel_limit CHANNEL_LIMIT
                        Sets the number of channels per layer you want to
                        calculate the gradient of.
  --images pattern [pattern ...]
			Absolute image path to the image. You can use wildcards.
  --outdir OUTDIR

For a list of absolute image paths call this script this way:

python extract_grad_map.py --images $(cat /path/to/imagelist.txt) --limit 1 --channel_limit 256 --layers probs pool5 --outdir /path/to/output/

The gradient maps are stored as Matlab .mat file and as png. In addition to these, the script also generates A html file to view the gradient maps and the input image. The gradient map is placed in the directory outdir/images'_parent_dir/image_filename/*. Be aware that approx. 45 MiB of storage is required per input image. For the whole CUB200-2011 dataset this means a total storage size of approx 800 GiB!

5. Part Localization

Apply the part localization using GMM fitting or maximum finding. Have a look in the part_localization folder for that. Open calcCUBPartLocs.m and adjust the paths. Now simply run calcCUBPartLocs(). This will create a file which has the same format as the part_locs.txt file of the CUB200-2011 dataset. You can use it for part-based classification.

6. Classification

We also provide the classification framework to use these part localizations and feature extraction with DeCAF. Go to the folder classification and open partEstimationDeepLearing.m. Have a look at line 40 and adjust the path such that it points to the correct file. Open settings.m and adjust the paths. Next, open settings.m and adjust the paths to liblinear and the virtual python environment. Now you can execute for example:

init
recRate = experimentParts('cub200_2011',200, struct('descriptor','plain','preprocessing_useMask','none','preprocessing_cropToBoundingbox',0), struct('partSelection',[1 2 3 9 14],'bothSymmetricParts',0,'descriptor','plain','trainPartLocation','est','preprocessing_relativePartSize',1.0/8,'preprocessing_cropToBoundingbox',0))

This will evaluate the classification performance on the standard train-test-split using the estimated part locations. Experiment parts has four parameters. The first one tell the function which dataset to use. You want to keep 'cub200_2011' here.

The second one is the number of classes to use, 3, 14 and 200 is supported here. Next is the setup for the global feature extraction. The only important setting is preprocessing_cropToBoundingbox. A value of 0 will tell the function not to use the ground truth bounding box during testing. You should leave the other two options as shown here.

The last one is the setup for the part features. You can select here, which parts you want to use and if you want to extract features from both symmetric parts, if both are visible. Since the part detector discovery associates some parts with the same channel, the location prediction will be the same for these. In this case, only select the parts which have unique channels here. In the example, the part 1, 2, 3, 9 and 14 are associated with different channels.

'trainPartLocation' tells the function, if grount-truth ('gt') or estimated ('est') part locations should be used for training. Since the discovered part detectors do not necessarily relate to semantic parts, 'est' usually is the better option here.

'preprocessing_relativePartSize' adjusts the size of patches, that are extracted at the estimated part locations. Please have a look at the paper for more information.

For the remaining options, you should keep everything as it is.

Acknowledgements

The classification framework is an extension of the excellent fine-grained recognition framework by Christoph Göring, Erik Rodner, Alexander Freytag and Joachim Denzler. You can find their project at https://github.com/cvjena/finegrained-cvpr2014.

Our work is based on DeCAF, a framework for convolutional neural networks. You can find the repository of the corresponding project at https://github.com/UCB-ICSI-Vision-Group/decaf-release/ .

License

Part Detector Discovery Framework by Marcel Simon, Erik Rodner and Joachim Denzler is licensed under the non-commercial license Creative Commons Attribution 4.0 International License. For usage beyond the scope of this license, please contact Marcel Simon.

You might also like...
Code for reproducing our analysis in the paper titled: Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agency
Code for reproducing our analysis in the paper titled: Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agency

Image Crop Analysis This is a repo for the code used for reproducing our Image Crop Analysis paper as shared on our blog post. If you plan to use this

Source code and dataset for ACL2021 paper: "ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning".

ERICA Source code and dataset for ACL2021 paper: "ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive L

Code for the ICML 2021 paper
Code for the ICML 2021 paper "Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptation", Haoxiang Wang, Han Zhao, Bo Li.

Bridging Multi-Task Learning and Meta-Learning Code for the ICML 2021 paper "Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Trainin

Data and Code for ACL 2021 Paper
Data and Code for ACL 2021 Paper "Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning"

Introduction Code and data for ACL 2021 Paper "Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning". We cons

Code for the paper "VisualBERT: A Simple and Performant Baseline for Vision and Language"

This repository contains code for the following two papers: VisualBERT: A Simple and Performant Baseline for Vision and Language (arxiv) with a short

Open source repository for the code accompanying the paper 'Non-Rigid Neural Radiance Fields Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video'.
Open source repository for the code accompanying the paper 'Non-Rigid Neural Radiance Fields Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video'.

Non-Rigid Neural Radiance Fields This is the official repository for the project "Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synt

Code for the Shortformer model, from the paper by Ofir Press, Noah A. Smith and Mike Lewis.

Shortformer This repository contains the code and the final checkpoint of the Shortformer model. This file explains how to run our experiments on the

Open source code for Paper
Open source code for Paper "A Co-Interactive Transformer for Joint Slot Filling and Intent Detection"

A Co-Interactive Transformer for Joint Slot Filling and Intent Detection This repository contains the PyTorch implementation of the paper: A Co-Intera

A code repository associated with the paper A Benchmark for Rough Sketch Cleanup by Chuan Yan, David Vanderhaeghe, and Yotam Gingold from SIGGRAPH Asia 2020.

A Benchmark for Rough Sketch Cleanup This is the code repository associated with the paper A Benchmark for Rough Sketch Cleanup by Chuan Yan, David Va

Releases(v1.0)
Owner
Computer Vision Group Jena
Computer Vision Group Jena
A PyTorch implementation for V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation

A PyTorch implementation of V-Net Vnet is a PyTorch implementation of the paper V-Net: Fully Convolutional Neural Networks for Volumetric Medical Imag

Matthew Macy 606 Dec 21, 2022
🦙 LaMa Image Inpainting, Resolution-robust Large Mask Inpainting with Fourier Convolutions, WACV 2022

🦙 LaMa Image Inpainting, Resolution-robust Large Mask Inpainting with Fourier Convolutions, WACV 2022

Advanced Image Manipulation Lab @ Samsung AI Center Moscow 4.7k Dec 31, 2022
Bottleneck Transformers for Visual Recognition

Bottleneck Transformers for Visual Recognition Experiments Model Params (M) Acc (%) ResNet50 baseline (ref) 23.5M 93.62 BoTNet-50 18.8M 95.11% BoTNet-

Myeongjun Kim 236 Jan 03, 2023
MapReader: A computer vision pipeline for the semantic exploration of maps at scale

MapReader A computer vision pipeline for the semantic exploration of maps at scale MapReader is an end-to-end computer vision (CV) pipeline designed b

Living with Machines 25 Dec 26, 2022
Based on the given clinical dataset, Predict whether the patient having Heart Disease or Not having Heart Disease

Heart_Disease_Classification Based on the given clinical dataset, Predict whether the patient having Heart Disease or Not having Heart Disease Dataset

Ashish 1 Jan 30, 2022
Level Based Customer Segmentation

level_based_customer_segmentation Level Based Customer Segmentation Persona Veri Seti kullanılarak müşteri segmentasyonu yapılmıştır. KOLONLAR : PRICE

Buse Yıldırım 6 Dec 21, 2021
Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation (CVPR 2021)

Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation (CVPR 2021, official Pytorch implementatio

Microsoft 247 Dec 25, 2022
WRENCH: Weak supeRvision bENCHmark

🔧 What is it? Wrench is a benchmark platform containing diverse weak supervision tasks. It also provides a common and easy framework for development

Jieyu Zhang 176 Dec 28, 2022
Multiple Object Tracking with Yolov5!

Tracking with yolov5 This implementation is for who need to tracking multi-object only with detector. You can easily track mult-object with your well

9 Nov 08, 2022
Fast, flexible and fun neural networks.

Brainstorm Discontinuation Notice Brainstorm is no longer being maintained, so we recommend using one of the many other,available frameworks, such as

IDSIA 1.3k Nov 21, 2022
ADSPM: Attribute-Driven Spontaneous Motion in Unpaired Image Translation

ADSPM: Attribute-Driven Spontaneous Motion in Unpaired Image Translation This repository provides a PyTorch implementation of ADSPM. Requirements Pyth

24 Jul 24, 2022
ConvMAE: Masked Convolution Meets Masked Autoencoders

ConvMAE ConvMAE: Masked Convolution Meets Masked Autoencoders Peng Gao1, Teli Ma1, Hongsheng Li2, Jifeng Dai3, Yu Qiao1, 1 Shanghai AI Laboratory, 2 M

Alpha VL Team of Shanghai AI Lab 345 Jan 08, 2023
Multiple Object Extraction from Aerial Imagery with Convolutional Neural Networks

This is an implementation of Volodymyr Mnih's dissertation methods on his Massachusetts road & building dataset and my original methods that are publi

Shunta Saito 255 Sep 07, 2022
Repository for self-supervised landmark discovery

self-supervised-landmarks Repository for self-supervised landmark discovery Requirements pytorch pynrrd (for 3d images) Usage The use of this models i

Riddhish Bhalodia 2 Apr 18, 2022
Official Implementation of 'UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformers' ICLR 2021(spotlight)

UPDeT Official Implementation of UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformers (ICLR 2021 spotlight) The

hhhusiyi 96 Dec 22, 2022
OBBDetection: an oriented object detection toolbox modified from MMdetection

OBBDetection note: If you have questions or good suggestions, feel free to propose issues and contact me. introduction OBBDetection is an oriented obj

MIXIAOXIN_HO 3 Nov 11, 2022
Python code to fuse multiple RGB-D images into a TSDF voxel volume.

Volumetric TSDF Fusion of RGB-D Images in Python This is a lightweight python script that fuses multiple registered color and depth images into a proj

Andy Zeng 845 Jan 03, 2023
Multi-task Self-supervised Object Detection via Recycling of Bounding Box Annotations (CVPR, 2019)

Multi-task Self-supervised Object Detection via Recycling of Bounding Box Annotations (CVPR 2019) To make better use of given limited labels, we propo

126 Sep 13, 2022
Face Recognition Attendance Project

Face-Recognition-Attendance-Project In This Project You will learn how to mark attendance using face recognition, Hello Guys This is Gautam Kumar, Thi

Gautam Kumar 1 Dec 03, 2022
Image Recognition using Pytorch

PyTorch Project Template A simple and well designed structure is essential for any Deep Learning project, so after a lot practice and contributing in

Sarat Chinni 1 Nov 02, 2021