Pytorch Feature Map Extractor

Overview

MapExtrackt

Downloads Release

Convolutional Neural Networks Are Beautiful

We all take our eyes for granted, we glance at an object for an instant and our brains can identify with ease. However distorted the information may be, we do a pretty good job at it.

Low light, obscured vision, poor eyesight... There are a myriad of situations where conditions are poor but still we manage to understand what an object is. Context helps, but we humans were created with sight in mind.

Computers have a harder time, but modern advances with convolutional neural networks are making this task a reality.

Computers are amazing, the neural networks and maps they create are beautiful.

Why not have an explore?


MapExtrakt makes viewing feature maps a breeze.

Catch a glimpse of how a computer can see.

MapExtrakt Usage


First import / gather your model (this does not have to be a pretrained pytorch model).

import torchvision
model = torchvision.models.vgg19(pretrained=True)

Import MapExtract's Feature Extractor and load in the model

from MapExtrackt import FeatureExtractor
fe = FeatureExtractor(model)

Set image to be analysed - input can be PIL Image, Numpy array or filepath. We are using the path

fe.set_image("pug.jpg")

View Layers

fe.display_from_map(layer_no=1)

Example Output

View Single Cells At a Time

fe.display_from_map(layer_no=2, cell_no=4)

Example Output

Slice the class to get a range of cells (Layer 2 Cells 0-9)

fe[2,0:10]

Example Output

Or Export Layers To Video

fe.write_video(out_size=(1200,800), file_name="output.avi", time_for_layer=60, transition_perc_layer=0.2)
MapExtrackt

More Examples

For LOTS more - view the jupyter notebook.

Examples


Installation

It's as easy as PyPI

pip install mapextrackt

or build from source in terminal

git clone https://github.com/lewis-morris/mapextrackt &&\
cd mapextrackt &&\
pip install -e .

Todo List

  • Add the ability to slice the class i.e FeatureExtractor[1,3]
  • Show parameters on the image
  • Fix video generation
  • Enable individual cells to be added to video
  • Add video parameters such as duration in seconds.
  • Clean up code
  • Make speed improvements

Author

Created by me, initially to view the outputs for my own pleasure.

If anyone has any suggestions or requests please send them over I'd be more than happy to consider.

[email protected]

You might also like...
Pytorch implementation of PSEnet with Pyramid Attention Network as feature extractor
Pytorch implementation of PSEnet with Pyramid Attention Network as feature extractor

Scene Text-Spotting based on PSEnet+CRNN Pytorch implementation of an end to end Text-Spotter with a PSEnet text detector and CRNN text recognizer. We

Replace MSFS2020's bing map to google map
Replace MSFS2020's bing map to google map

English verison here 中文 免责声明 本教程提到的方法仅用于研究和学习用途。我不对使用、拓展该教程及方法所造成的任何法律责任和损失负责。 背景 微软模拟飞行2020的地景使用了Bing的卫星地图,然而卫星地图比较老旧,很多地区都是几年前的图设置直接是没有的。这种现象在全球不同地区

An extension for asreview implements a version of the tf-idf feature extractor that saves the matrix and the vocabulary.

Extension - matrix and vocabulary extractor for TF-IDF and Doc2Vec An extension for ASReview that adds a tf-idf extractor that saves the matrix and th

Implementation of the Triangle Multiplicative module, used in Alphafold2 as an efficient way to mix rows or columns of a 2d feature map, as a standalone package for Pytorch
Implementation of the Triangle Multiplicative module, used in Alphafold2 as an efficient way to mix rows or columns of a 2d feature map, as a standalone package for Pytorch

Triangle Multiplicative Module - Pytorch Implementation of the Triangle Multiplicative module, used in Alphafold2 as an efficient way to mix rows or c

Official implementation of "StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation" (SIGGRAPH 2021)

StyleCariGAN in PyTorch Official implementation of StyleCariGAN:Caricature Generation via StyleGAN Feature Map Modulation in PyTorch Requirements PyTo

Official implementation of
Official implementation of "StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation" (SIGGRAPH 2021)

StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation This repository contains the official PyTorch implementation of the following

VCM EE1.2 P-layer feature map anchor generation 137th MPEG-VCM

VCM EE1.2 P-layer feature map anchor generation 137th MPEG-VCM

RetinaNet-PyTorch - A RetinaNet Pytorch Implementation on remote sensing images and has the similar mAP result with RetinaNet in MMdetection
Html Content / Article Extractor, web scrapping lib in Python

Python-Goose - Article Extractor Intro Goose was originally an article extractor written in Java that has most recently (Aug2011) been converted to a

CUTIE (TensorFlow implementation of Convolutional Universal Text Information Extractor)
CUTIE (TensorFlow implementation of Convolutional Universal Text Information Extractor)

CUTIE TensorFlow implementation of the paper "CUTIE: Learning to Understand Documents with Convolutional Universal Text Information Extractor." Xiaohu

A sketch extractor for anime/illustration.
A sketch extractor for anime/illustration.

Anime2Sketch Anime2Sketch: A sketch extractor for illustration, anime art, manga By Xiaoyu Xiang Updates 2021.5.2: Upload more example results of anim

Widevine KEY Extractor in Python

Widevine Client 3 This was originally written by T3rry7f. This repo is slightly modified version of his repo. This only works on standard Windows! Usa

Cobalt Strike Beacon configuration extractor and parser.

Cobalt Strike Configuration Extractor and Parser Overview Pure Python library and set of scripts to extract and parse configurations (configs) from Co

Advance Image Downloader/Extractor (Job) is a Python-Flask web-based app, which will help the user download the any kind of Images at any date and time over the internet. These images will get downloaded as a job and then let user know that the images have been downloaded by sending them a link over an email. Configuration Extractor for EXE4J PE files
Configuration Extractor for EXE4J PE files

EXE4J Configuration Extractor This script helps reverse engineering Portable Executable files created with EXE4J by extracting their configuration dat

A string extractor module for python

A string extractor module for python

A telegram string extractor bot

Made with Python3 (C) @FayasNoushad Copyright permission under MIT License License - https://github.com/FayasNoushad/String-Extract-Bot/blob/main/LIC

A telegram user and chat info extractor with pyrogram python module

Made with Python3 (C) @FayasNoushad Copyright permission under MIT License License - https://github.com/FayasNoushad/Telegram-Info/blob/main/LICENSE

Tesla App Update Differences Extractor

Tesla App Update Differences Extractor Python program that finds the differences between two versions of the Tesla App. When Tesla updates the app a l

Comments
  • UnboundLocalError: local variable 'font_loc' referenced before assignment

    UnboundLocalError: local variable 'font_loc' referenced before assignment


    UnboundLocalError Traceback (most recent call last) in ----> 1 fe.display_from_map(layer_no=2, cell_no=4)

    ~/SageMaker/speedlimit/mapextrackt/MapExtrackt/extractor.py in display_from_map(self, layer_no, cell_no, out_type, colourize, outsize, border, picture_in_picture, write_text) 194 if self.write_text.lower() != "none": 195 subtext = "" --> 196 img = self._write_text(img, layer_no, cell_no) 197 198 if self.out_type.lower() == "pil":

    ~/SageMaker/speedlimit/mapextrackt/MapExtrackt/extractor.py in _write_text(self, img, layer_no, cell_no, flip) 242 if flip: 243 text, subtext = subtext, text --> 244 img = draw_text(img, text, subtext) 245 246 return img

    ~/SageMaker/speedlimit/mapextrackt/MapExtrackt/functions.py in draw_text(img, text, subtext) 194 195 --> 196 font = ImageFont.truetype(font_loc, size) 197 smaller_font = ImageFont.truetype(font_loc, smaller_size) 198

    UnboundLocalError: local variable 'font_loc' referenced before assignment

    opened by RohitKeshari 3
  • Error on display_from_map in OpenCV (3.4.2)

    Error on display_from_map in OpenCV (3.4.2)

    EDIT: SORRY MISSED VERSIONS REQUIREMENTS, WILL FIX THEM AND SEE IF ITS CORRECTED I'm using resnet18 with last layer softmax classifier and getting this error. By the way thank you for your work 👍

    `from MapExtrackt import FeatureExtractor
    
    fe = FeatureExtractor(model_conv)
    
    fe.set_image('im2.jpg')
    
    fe.display_from_map(layer_no=1) #Tried several just in case: Layers -> 0-72`
    

    ERROR:

    `---------------------------------------------------------------------------
    error                                     Traceback (most recent call last)
     in 
          5 fe.set_image('im2.jpg')
          6 
    ----> 7 fe.display_from_map(layer_no=1)
    
    C:\ProgramData\Anaconda3\envs\vc\lib\site-packages\MapExtrackt\extractor.py in display_from_map(self, layer_no, cell_no, out_type, colourize, outsize, border, picture_in_picture, write_text)
        179 
        180         # get image map
    --> 181         img = self.__return_feature_map(layer_no, single=cell_no)
        182 
        183         # return type
    
    C:\ProgramData\Anaconda3\envs\vc\lib\site-packages\MapExtrackt\extractor.py in __return_feature_map(self, layer_no, single)
        726                 # if colourize
        727                 if self.colourize > -1:
    --> 728                     img = colourize_image(img, self.colourize)
        729 
        730                 # if border pad
    
    C:\ProgramData\Anaconda3\envs\vc\lib\site-packages\MapExtrackt\functions.py in colourize_image(img, colour_type)
        168         return base
        169     else:
    --> 170         return cv2.applyColorMap(img, colour_type, None)
        171 
        172 
    
    error: OpenCV(3.4.2) C:\Miniconda3\conda-bld\opencv-suite_1534379934306\work\modules\imgproc\src\colormap.cpp:558: error: (-5:Bad argument) Unknown colormap id; use one of COLORMAP_* in function 'cv::applyColorMap'`
    
    opened by PGG-DeepAI 3
  • Feature request: Possibility to visualize the input of a specific layer

    Feature request: Possibility to visualize the input of a specific layer

    Feature request: Possibility to visualize the input of a specific layer. It is useful if you want to visualize the activations (weights after the activation function, e.g., Relu) because the inputs to the next layer after the Relu are effectively the activation.

    opened by neoglez 1
  • Question: Is it intentional to not divide by the max-min difference when normalizing layer?

    Question: Is it intentional to not divide by the max-min difference when normalizing layer?

    Thanks for the great library.

    When setting the image, one options is to normalize the layer for viewing (parameter normalize_layer), if normalize_layer=False all features (independently of the others in the same layer) are normalized using min-max feature scaling. When normalize_layer=True, one would expect the layer to be normalized across all features using the same strategy, namely min-max feature scaling. However, the denominator doesn't contain the difference max-min but only max. Is this intentioned? If yes, why?

    opened by neoglez 2
Releases(0.4.8.4)
Owner
Lewis Morris
I'm just here to learn, experiment and enjoy.
Lewis Morris
Visual Computing Group (Ulm University) 99 Nov 30, 2022
Neural network visualization toolkit for tf.keras

Neural network visualization toolkit for tf.keras

Yasuhiro Kubota 262 Dec 19, 2022
Quickly and easily create / train a custom DeepDream model

Dream-Creator This project aims to simplify the process of creating a custom DeepDream model by using pretrained GoogleNet models and custom image dat

56 Jan 03, 2023
FairML - is a python toolbox auditing the machine learning models for bias.

======== FairML: Auditing Black-Box Predictive Models FairML is a python toolbox auditing the machine learning models for bias. Description Predictive

Julius Adebayo 338 Nov 09, 2022
👋🦊 Xplique is a Python toolkit dedicated to explainability, currently based on Tensorflow.

👋🦊 Xplique is a Python toolkit dedicated to explainability, currently based on Tensorflow.

DEEL 343 Jan 02, 2023
A collection of infrastructure and tools for research in neural network interpretability.

Lucid Lucid is a collection of infrastructure and tools for research in neural network interpretability. We're not currently supporting tensorflow 2!

4.5k Jan 07, 2023
Pytorch Feature Map Extractor

MapExtrackt Convolutional Neural Networks Are Beautiful We all take our eyes for granted, we glance at an object for an instant and our brains can ide

Lewis Morris 40 Dec 07, 2022
A ultra-lightweight 3D renderer of the Tensorflow/Keras neural network architectures

A ultra-lightweight 3D renderer of the Tensorflow/Keras neural network architectures

Souvik Pratiher 16 Nov 17, 2021
Using / reproducing ACD from the paper "Hierarchical interpretations for neural network predictions" 🧠 (ICLR 2019)

Hierarchical neural-net interpretations (ACD) 🧠 Produces hierarchical interpretations for a single prediction made by a pytorch neural network. Offic

Chandan Singh 111 Jan 03, 2023
A library for debugging/inspecting machine learning classifiers and explaining their predictions

ELI5 ELI5 is a Python package which helps to debug machine learning classifiers and explain their predictions. It provides support for the following m

2.6k Dec 30, 2022
Algorithms for monitoring and explaining machine learning models

Alibi is an open source Python library aimed at machine learning model inspection and interpretation. The focus of the library is to provide high-qual

Seldon 1.9k Dec 30, 2022
Portal is the fastest way to load and visualize your deep neural networks on images and videos 🔮

Portal is the fastest way to load and visualize your deep neural networks on images and videos 🔮

Datature 243 Jan 05, 2023
Code for visualizing the loss landscape of neural nets

Visualizing the Loss Landscape of Neural Nets This repository contains the PyTorch code for the paper Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer

Tom Goldstein 2.2k Dec 30, 2022
Interactive convnet features visualization for Keras

Quiver Interactive convnet features visualization for Keras The quiver workflow Video Demo Build your model in keras model = Model(...) Launch the vis

Keplr 1.7k Dec 21, 2022
JittorVis - Visual understanding of deep learning model.

JittorVis - Visual understanding of deep learning model.

182 Jan 06, 2023
Tool for visualizing attention in the Transformer model (BERT, GPT-2, Albert, XLNet, RoBERTa, CTRL, etc.)

Tool for visualizing attention in the Transformer model (BERT, GPT-2, Albert, XLNet, RoBERTa, CTRL, etc.)

Jesse Vig 4.7k Jan 01, 2023
pytorch implementation of "Distilling a Neural Network Into a Soft Decision Tree"

Soft-Decision-Tree Soft-Decision-Tree is the pytorch implementation of Distilling a Neural Network Into a Soft Decision Tree, paper recently published

Kim Heecheol 262 Dec 04, 2022
Bias and Fairness Audit Toolkit

The Bias and Fairness Audit Toolkit Aequitas is an open-source bias audit toolkit for data scientists, machine learning researchers, and policymakers

Data Science for Social Good 513 Jan 06, 2023
Pytorch implementation of convolutional neural network visualization techniques

Convolutional Neural Network Visualizations This repository contains a number of convolutional neural network visualization techniques implemented in

Utku Ozbulak 7k Jan 03, 2023
Visualizer for neural network, deep learning, and machine learning models

Netron is a viewer for neural network, deep learning and machine learning models. Netron supports ONNX, TensorFlow Lite, Keras, Caffe, Darknet, ncnn,

Lutz Roeder 20.9k Dec 28, 2022