Spatial Transformer Nets in TensorFlow/ TensorLayer

Last update: Nov 23, 2022

Overview

MOVED TO HERE

Spatial Transformer Networks

Spatial Transformer Networks (STN) is a dynamic mechanism that produces transformations of input images (or feature maps)including scaling, cropping, rotations, as well as non-rigid deformations. This enables the network to not only select regions of an image that are most relevant (attention), but also to transform those regions to simplify recognition in the following layers.

Video for different transformation click me.

In this repositary, we implemented a STN for 2D Affine Transformation on MNIST dataset. We generated images with size of 40x40 from the original MNIST dataset, and distorted the images by random rotation, shifting, shearing and zoom in/out. The STN was able to learn to automatically apply transformations on distorted images via classification task.

Fig 1：Transformation

Fig 2：Network

Fig 3：Formula

Result

After classification task, the STN is able to transform the distorted image from Fig 4 back to Fig 5.

Fig 4: Input

Fig 5: Output

Comments

Export graph

Hello,

I'm trying to export this model and its weights into a frozen graph. So far I did this in the "save images" part of the training loop:

saver = tf.train.Saver() saver.save(sess, 'my_stn_model_' + str(epoch)) tf.train.write_graph(sess.graph_def, ".", "test.pb", False) #proto

Then I have my pb and the weights. But I am unable to generate a frozen graph from this because I cannot guess the output node in the graph (which is a really complex one it seems).

I attached the pb which I am able to visualize using Netron:

test.zip

Thanks in advance.

opened by hvico 0
Unexpected Output

I'm trying to train the STN. The training dataset which I've provided contains MNIST dataset images which are rotated 90 degree and the testing dataset contains simple MNIST dataset images. The output I'm getting is straight MNIST images. I was expecting the output to contain characters which are rotated 90 degrees. Can you please guide me on how this works?

opened by nabeel3133 4

Spatial Transformer Nets in TensorFlow/ TensorLayer

Related tags

Overview

MOVED TO HERE

Spatial Transformer Networks

Result

You might also like...

Group Activity Recognition with Clustered Spatial Temporal Transformer

VSR-Transformer - This paper proposes a new Transformer for video super-resolution (called VSR-Transformer).

Python implementation of cover trees, near-drop-in replacement for scipy.spatial.kdtree

data/code repository of "C2F-FWN: Coarse-to-Fine Flow Warping Network for Spatial-Temporal Consistent Motion Transfer"

Unofficial implementation of "TTNet: Real-time temporal and spatial video analysis of table tennis" (CVPR 2020)

Official PyTorch implementation of Spatial Dependency Networks.

Spatial Intention Maps for Multi-Agent Mobile Manipulation (ICRA 2021)

The open source code of SA-UNet: Spatial Attention U-Net for Retinal Vessel Segmentation.

Spatial Action Maps for Mobile Manipulation (RSS 2020)

Comments

Export graph

Unexpected Output

Releases(0.0.1)

0.0.1(Jun 26, 2017)

Owner

Hao

Simple helper library to convert a collection of numpy data to tfrecord, and build a tensorflow dataset from the tfrecord.

MXNet implementation for: Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution

Unoffical reMarkable AddOn for Firefox.

Python lib to talk to pylontech lithium batteries (US2000, US3000, ...) using RS485

An Implementation of SiameseRPN with Feature Pyramid Networks

VoxHRNet - Whole Brain Segmentation with Full Volume Neural Network

B-cos Networks: Attention is All we Need for Interpretability

Annotated, understandable, and visually interpretable PyTorch implementations of: VAE, BIRVAE, NSGAN, MMGAN, WGAN, WGANGP, LSGAN, DRAGAN, BEGAN, RaGAN, InfoGAN, fGAN, FisherGAN

Scaling and Benchmarking Self-Supervised Visual Representation Learning

[AAAI 2022] Separate Contrastive Learning for Organs-at-Risk and Gross-Tumor-Volume Segmentation with Limited Annotation

[ICLR 2022 Oral] F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantization

The official implementation of CVPR 2021 Paper: Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation.

The devkit of the nuScenes dataset.

Cross Quality LFW: A database for Analyzing Cross-Resolution Image Face Recognition in Unconstrained Environments

Embracing Single Stride 3D Object Detector with Sparse Transformer

Example scripts for the detection of lanes using the ultra fast lane detection model in Tensorflow Lite.

A font family with a great monospaced variant for programmers.

Unofficial PyTorch implementation of Attention Free Transformer (AFT) layers by Apple Inc.

This is an official implementation for "SimMIM: A Simple Framework for Masked Image Modeling".

PyElastica is the Python implementation of Elastica, an open-source software for the simulation of assemblies of slender, one-dimensional structures using Cosserat Rod theory.