Offical implementation of Shunted Self-Attention via Multi-Scale Token Aggregation

Last update: Dec 27, 2022

Related tags

Overview

Shunted Transformer

This is the offical implementation of Shunted Self-Attention via Multi-Scale Token Aggregation by Sucheng Ren, Daquan Zhou, Shengfeng He, Jiashi Feng, Xinchao Wang

Training from scratch

Training

bash dist_train.sh

Model Zoo

The checkpoints can be found at Goolge Drive, Baidu Pan (code:hazr) (Checkpoints of the large models are coming soon.)

Method	Size	[email protected]	#Params (M)
Shunted-T	224	79.8	11.5
Shunted-S	224	82.9	22.4
Shunted-B	224	84.0	39.6

Citation

@misc{ren2021shunted,
      title={Shunted Self-Attention via Multi-Scale Token Aggregation}, 
      author={Sucheng Ren and Daquan Zhou and Shengfeng He and Jiashi Feng and Xinchao Wang},
      year={2021},
      eprint={2111.15193},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Owner

First-year master student at SCUT

GitHub Repository

Official re-implementation of the Calibrated Adversarial Refinement model described in the paper Calibrated Adversarial Refinement for Stochastic Semantic Segmentation

31 Nov 22, 2022

Code for Max-Margin Contrastive Learning - AAAI 2022

Max-Margin Contrastive Learning This is a pytorch implementation for the paper Max-Margin Contrastive Learning accepted to AAAI 2022. This repository

12 Oct 22, 2022

A pyparsing-based library for parsing SOQL statements

CONTRIBUTORS WANTED!! Installation pip install python-soql-parser or, with poetry poetry add python-soql-parser Usage from python_soql_parser import p

0 Jun 07, 2022

This repo provides the official code for TransBTS: Multimodal Brain Tumor Segmentation Using Transformer (https://arxiv.org/pdf/2103.04430.pdf).

TransBTS: Multimodal Brain Tumor Segmentation Using Transformer This repo is the official implementation for TransBTS: Multimodal Brain Tumor Segmenta

247 Dec 28, 2022

Offical implementation of Shunted Self-Attention via Multi-Scale Token Aggregation

Related tags

Overview

Shunted Transformer

Training from scratch

Training

Model Zoo

Citation

Owner

Official re-implementation of the Calibrated Adversarial Refinement model described in the paper Calibrated Adversarial Refinement for Stochastic Semantic Segmentation

Code for Max-Margin Contrastive Learning - AAAI 2022

A pyparsing-based library for parsing SOQL statements

This repo provides the official code for TransBTS: Multimodal Brain Tumor Segmentation Using Transformer (https://arxiv.org/pdf/2103.04430.pdf).

SE3 Pose Interp - Interpolate camera pose or trajectory in SE3, pose interpolation, trajectory interpolation

Reimplementation of NeurIPS'19: "Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting" by Shu et al.

Implement slightly different caffe-segnet in tensorflow

Official implementation of the paper ``Unifying Nonlocal Blocks for Neural Networks'' (ICCV'21)

Trajectory Prediction with Graph-based Dual-scale Context Fusion

WORD: Revisiting Organs Segmentation in the Whole Abdominal Region

An Ensemble of CNN (Python 3.5.1 Tensorflow 1.3 numpy 1.13)

Public implementation of "Learning from Suboptimal Demonstration via Self-Supervised Reward Regression" from CoRL'21

DeepFashion2 is a comprehensive fashion dataset.

Implementation of the famous Image Manipulation\Forgery Detector "ManTraNet" in Pytorch

CLIP (Contrastive Language–Image Pre-training) for Italian

Probabilistic Programming and Statistical Inference in PyTorch

Scalable, event-driven, deep-learning-friendly backtesting library

An Industrial Grade Federated Learning Framework

Distributing reference energies for SMIRNOFF implementations

Multi-Horizon-Forecasting-for-Limit-Order-Books