Offical implementation of Shunted Self-Attention via Multi-Scale Token Aggregation

Last update: Dec 27, 2022

Related tags

Overview

Shunted Transformer

This is the offical implementation of Shunted Self-Attention via Multi-Scale Token Aggregation by Sucheng Ren, Daquan Zhou, Shengfeng He, Jiashi Feng, Xinchao Wang

Training from scratch

Training

bash dist_train.sh

Model Zoo

The checkpoints can be found at Goolge Drive, Baidu Pan (code:hazr) (Checkpoints of the large models are coming soon.)

Method	Size	[email protected]	#Params (M)
Shunted-T	224	79.8	11.5
Shunted-S	224	82.9	22.4
Shunted-B	224	84.0	39.6

Citation

@misc{ren2021shunted,
      title={Shunted Self-Attention via Multi-Scale Token Aggregation}, 
      author={Sucheng Ren and Daquan Zhou and Shengfeng He and Jiashi Feng and Xinchao Wang},
      year={2021},
      eprint={2111.15193},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Owner

First-year master student at SCUT

GitHub Repository

A large-scale video dataset for the training and evaluation of 3D human pose estimation models

ASPset-510 ASPset-510 (Australian Sports Pose Dataset) is a large-scale video dataset for the training and evaluation of 3D human pose estimation mode

36 Oct 30, 2022

Mercer Gaussian Process (MGP) and Fourier Gaussian Process (FGP) Regression

Mercer Gaussian Process (MGP) and Fourier Gaussian Process (FGP) Regression We provide the code used in our paper "How Good are Low-Rank Approximation

0 Dec 13, 2021

HMLLDB is a collection of LLDB commands to assist in the debugging of iOS apps.

HMLLDB is a collection of LLDB commands to assist in the debugging of iOS apps. 中文介绍 Features Non-intrusive. Your iOS project does not need to be modi

47 Oct 22, 2022

Deep Multi-Magnification Network for multi-class tissue segmentation of whole slide images

Deep Multi-Magnification Network This repository provides training and inference codes for Deep Multi-Magnification Network published here. Deep Multi

12 Aug 06, 2022

Joint Unsupervised Learning (JULE) of Deep Representations and Image Clusters.

Joint Unsupervised Learning (JULE) of Deep Representations and Image Clusters. Overview This project is a Torch implementation for our CVPR 2016 paper

278 Dec 25, 2022

Pytorch implementation of the Variational Recurrent Neural Network (VRNN).

VariationalRecurrentNeuralNetwork Pytorch implementation of the Variational RNN (VRNN), from A Recurrent Latent Variable Model for Sequential Data. Th

251 Dec 17, 2022

Novel Instances Mining with Pseudo-Margin Evaluation for Few-Shot Object Detection

Novel Instances Mining with Pseudo-Margin Evaluation for Few-Shot Object Detection (NimPme) The official implementation of Novel Instances Mining with

12 Sep 08, 2022

A Dynamic Residual Self-Attention Network for Lightweight Single Image Super-Resolution

DRSAN A Dynamic Residual Self-Attention Network for Lightweight Single Image Super-Resolution Karam Park, Jae Woong Soh, and Nam Ik Cho Environments U

4 May 10, 2022

Tensorflow implementation of Semi-supervised Sequence Learning (https://arxiv.org/abs/1511.01432)

Transfer Learning for Text Classification with Tensorflow Tensorflow implementation of Semi-supervised Sequence Learning(https://arxiv.org/abs/1511.01

82 Oct 22, 2022

Cache Requests in Deta Bases and Echo them with Deta Micros

Deta Echo Cache Leverage the awesome Deta Micros and Deta Base to cache requests and echo them as needed. Stop worrying about slow public APIs or agre

8 Dec 07, 2021

We provided a matlab implementation for an evolutionary multitasking AUC optimization framework (EMTAUC).

EMTAUC We provided a matlab implementation for an evolutionary multitasking AUC optimization framework (EMTAUC). In this code, SBGA is considered a ba

7 Nov 24, 2022

code for paper "Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning" by Zhongzheng Ren, Raymond A. Yeh, Alexander G. Schwing.

Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning Overview This code is for paper: Not All Unlabeled Data are Equa

22 Nov 23, 2022

CR-Fill: Generative Image Inpainting with Auxiliary Contextual Reconstruction. ICCV 2021

182 Dec 20, 2022

traiNNer is an open source image and video restoration (super-resolution, denoising, deblurring and others) and image to image translation toolbox based on PyTorch.

traiNNer traiNNer is an open source image and video restoration (super-resolution, denoising, deblurring and others) and image to image translation to

202 Jan 04, 2023

Offical implementation of Shunted Self-Attention via Multi-Scale Token Aggregation

Related tags

Overview

Shunted Transformer

Training from scratch

Training

Model Zoo

Citation

Owner

A large-scale video dataset for the training and evaluation of 3D human pose estimation models

Mercer Gaussian Process (MGP) and Fourier Gaussian Process (FGP) Regression

HMLLDB is a collection of LLDB commands to assist in the debugging of iOS apps.

Deep Multi-Magnification Network for multi-class tissue segmentation of whole slide images

Joint Unsupervised Learning (JULE) of Deep Representations and Image Clusters.

Pytorch implementation of the Variational Recurrent Neural Network (VRNN).

Novel Instances Mining with Pseudo-Margin Evaluation for Few-Shot Object Detection

A Dynamic Residual Self-Attention Network for Lightweight Single Image Super-Resolution

Tensorflow implementation of Semi-supervised Sequence Learning (https://arxiv.org/abs/1511.01432)

Cache Requests in Deta Bases and Echo them with Deta Micros

We provided a matlab implementation for an evolutionary multitasking AUC optimization framework (EMTAUC).

code for paper "Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning" by Zhongzheng Ren, Raymond A. Yeh, Alexander G. Schwing.

CR-Fill: Generative Image Inpainting with Auxiliary Contextual Reconstruction. ICCV 2021

traiNNer is an open source image and video restoration (super-resolution, denoising, deblurring and others) and image to image translation toolbox based on PyTorch.

Links to works on deep learning algorithms for physics problems, TUM-I15 and beyond

An efficient implementation of GPNN

PyTorch implementation for "Mining Latent Structures with Contrastive Modality Fusion for Multimedia Recommendation"

Contains code for Deep Kernelized Dense Geometric Matching

An implementation of the AlphaZero algorithm for Gomoku (also called Gobang or Five in a Row)

RepVGG: Making VGG-style ConvNets Great Again

Offical implementation of Shunted Self-Attention via Multi-Scale Token Aggregation

Related tags

Overview

Shunted Transformer

Training from scratch

Training

Model Zoo

Citation

Owner

A large-scale video dataset for the training and evaluation of 3D human pose estimation models

Mercer Gaussian Process (MGP) and Fourier Gaussian Process (FGP) Regression

HMLLDB is a collection of LLDB commands to assist in the debugging of iOS apps.

Deep Multi-Magnification Network for multi-class tissue segmentation of whole slide images

Joint Unsupervised Learning (JULE) of Deep Representations and Image Clusters.

Pytorch implementation of the Variational Recurrent Neural Network (VRNN).

Novel Instances Mining with Pseudo-Margin Evaluation for Few-Shot Object Detection

A Dynamic Residual Self-Attention Network for Lightweight Single Image Super-Resolution

Tensorflow implementation of Semi-supervised Sequence Learning (https://arxiv.org/abs/1511.01432)

Cache Requests in Deta Bases and Echo them with Deta Micros

We provided a matlab implementation for an evolutionary multitasking AUC optimization framework (EMTAUC).

code for paper "Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning" by Zhongzheng Ren*, Raymond A. Yeh*, Alexander G. Schwing.

CR-Fill: Generative Image Inpainting with Auxiliary Contextual Reconstruction. ICCV 2021

traiNNer is an open source image and video restoration (super-resolution, denoising, deblurring and others) and image to image translation toolbox based on PyTorch.

Links to works on deep learning algorithms for physics problems, TUM-I15 and beyond

An efficient implementation of GPNN

PyTorch implementation for "Mining Latent Structures with Contrastive Modality Fusion for Multimedia Recommendation"

Contains code for Deep Kernelized Dense Geometric Matching

An implementation of the AlphaZero algorithm for Gomoku (also called Gobang or Five in a Row)

RepVGG: Making VGG-style ConvNets Great Again

code for paper "Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning" by Zhongzheng Ren, Raymond A. Yeh, Alexander G. Schwing.