Unofficial PyTorch implementation of Fastformer based on paper "Fastformer: Additive Attention Can Be All You Need"."

Last update: Dec 06, 2022

Related tags

Overview

Fastformer-PyTorch

Unofficial PyTorch implementation of Fastformer based on paper Fastformer: Additive Attention Can Be All You Need.

Usage :

import torch
import Fastformer

model = Fastformer(dim = 3, decode_dim = 8)
x = torch.randn(4, 6, 3)
mask = torch.ones(1, 8).bool()
result = model(x, mask)
print(result.size())

Citation :

@misc{wu2021fastformer,
    title={Fastformer: Additive Attention Can Be All You Need},
    author={Chuhan Wu, Fangzhao Wu, Tao Qi and Yongfeng Huang},
    year={2021},
    eprint={2108.09084v2},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

If this implement have any problem please let me know, thank you.

Owner

Hong-Jia Chen

Master student at National Chung Cheng University, Taiwan. Interested in Deep Learning and Computer Vision.

GitHub Repository

Hcaptcha-challenger - Gracefully face hCaptcha challenge with Yolov5(ONNX) embedded solution

hCaptcha Challenger 🚀 Gracefully face hCaptcha challenge with Yolov5(ONNX) embe

593 Jan 03, 2023

ATAC: Adversarially Trained Actor Critic

ATAC: Adversarially Trained Actor Critic Adversarially Trained Actor Critic for Offline Reinforcement Learning by Ching-An Cheng*, Tengyang Xie*, Nan

41 Dec 08, 2022

Official repository for Fourier model that can generate periodic signals

Conditional Generation of Periodic Signals with Fourier-Based Decoder Jiyoung Lee, Wonjae Kim, Daehoon Gwak, Edward Choi This repository provides offi

8 May 25, 2022

Contrastive Feature Loss for Image Prediction

Contrastive Feature Loss for Image Prediction We provide a PyTorch implementation of our contrastive feature loss presented in: Contrastive Feature Lo

44 Oct 05, 2022

Fast and Easy Infinite Neural Networks in Python

1.9k Jan 09, 2023

A general framework for inferring CNNs efficiently. Reduce the inference latency of MobileNet-V3 by 1.3x on an iPhone XS Max without sacrificing accuracy.

GFNet-Pytorch (NeurIPS 2020) This repo contains the official code and pre-trained models for the glance and focus network (GFNet). Glance and Focus: a

169 Oct 28, 2022

An open source python library for automated feature engineering

"One of the holy grails of machine learning is to automate more and more of the feature engineering process." ― Pedro Domingos, A Few Useful Things to

6.4k Jan 03, 2023

Reviatalizing Optimization for 3D Human Pose and Shape Estimation: A Sparse Constrained Formulation

Reviatalizing Optimization for 3D Human Pose and Shape Estimation: A Sparse Constrained Formulation This is the implementation of the approach describ

47 Nov 15, 2022

HAT: Hierarchical Aggregation Transformers for Person Re-identification

11 Sep 05, 2022

Segmentation models with pretrained backbones. PyTorch.

Python library with Neural Networks for Image Segmentation based on PyTorch. The main features of this library are: High level API (just two lines to

6.6k Jan 06, 2023

joint detection and semantic segmentation, based on ultralytics/yolov5,

Multi YOLO V5——Detection and Semantic Segmentation Overeview This is my undergraduate graduation project which based on ultralytics YOLO V5 tag v5.0.

477 Jan 06, 2023

Elastic weight consolidation technique for incremental learning.

Overcoming-Catastrophic-forgetting-in-Neural-Networks Elastic weight consolidation technique for incremental learning. About Use this API if you dont

89 Dec 22, 2022

Official implementation of Self-supervised Image-to-text and Text-to-image Synthesis

Self-supervised Image-to-text and Text-to-image Synthesis This is the official implementation of Self-supervised Image-to-text and Text-to-image Synth

6 Jul 31, 2022

Block Sparse movement pruning

Movement Pruning: Adaptive Sparsity by Fine-Tuning Magnitude pruning is a widely used strategy for reducing model size in pure supervised learning; ho

54 Dec 20, 2022

A2LP for short, ECCV2020 spotlight, Investigating SSL principles for UDA problems

Label-Propagation-with-Augmented-Anchors (A2LP) Official codes of the ECCV2020 spotlight (label propagation with augmented anchors: a simple semi-supe

20 Oct 27, 2022

🔎 Super-scale your images and run experiments with Residual Dense and Adversarial Networks.

Image Super-Resolution (ISR) The goal of this project is to upscale and improve the quality of low resolution images. This project contains Keras impl

4k Jan 08, 2023

Qcover is an open source effort to help exploring combinatorial optimization problems in Noisy Intermediate-scale Quantum(NISQ) processor.

Qcover is an open source effort to help exploring combinatorial optimization problems in Noisy Intermediate-scale Quantum(NISQ) processor. It is devel

33 Nov 11, 2022

Unofficial PyTorch implementation of Fastformer based on paper "Fastformer: Additive Attention Can Be All You Need"."

Related tags

Overview

Fastformer-PyTorch

Usage :

Citation :

If this implement have any problem please let me know, thank you.

Owner

Hong-Jia Chen

Hcaptcha-challenger - Gracefully face hCaptcha challenge with Yolov5(ONNX) embedded solution

ATAC: Adversarially Trained Actor Critic

Official repository for Fourier model that can generate periodic signals

Contrastive Feature Loss for Image Prediction

Fast and Easy Infinite Neural Networks in Python

A general framework for inferring CNNs efficiently. Reduce the inference latency of MobileNet-V3 by 1.3x on an iPhone XS Max without sacrificing accuracy.

An open source python library for automated feature engineering

Reviatalizing Optimization for 3D Human Pose and Shape Estimation: A Sparse Constrained Formulation

HAT: Hierarchical Aggregation Transformers for Person Re-identification

Segmentation models with pretrained backbones. PyTorch.

joint detection and semantic segmentation, based on ultralytics/yolov5,

Elastic weight consolidation technique for incremental learning.

Official implementation of Self-supervised Image-to-text and Text-to-image Synthesis

Block Sparse movement pruning

A2LP for short, ECCV2020 spotlight, Investigating SSL principles for UDA problems

🔎 Super-scale your images and run experiments with Residual Dense and Adversarial Networks.

Qcover is an open source effort to help exploring combinatorial optimization problems in Noisy Intermediate-scale Quantum(NISQ) processor.

Portfolio Optimization and Quantitative Strategic Asset Allocation in Python

Chinese Mandarin tts text-to-speech 中文 (普通话) 语音合成 , by fastspeech 2 , implemented in pytorch, using waveglow as vocoder,

Task-based end-to-end model learning in stochastic optimization

Unofficial PyTorch implementation of Fastformer based on paper "Fastformer: Additive Attention Can Be All You Need"."

Related tags

Overview

Fastformer-PyTorch

Usage :

Citation :

If this implement have any problem please let me know, thank you.

Owner

Hong-Jia Chen

Hcaptcha-challenger - Gracefully face hCaptcha challenge with Yolov5(ONNX) embedded solution

ATAC: Adversarially Trained Actor Critic

Official repository for Fourier model that can generate periodic signals

Contrastive Feature Loss for Image Prediction

Fast and Easy Infinite Neural Networks in Python

A general framework for inferring CNNs efficiently. Reduce the inference latency of MobileNet-V3 by 1.3x on an iPhone XS Max without sacrificing accuracy.

An open source python library for automated feature engineering

Reviatalizing Optimization for 3D Human Pose and Shape Estimation: A Sparse Constrained Formulation

HAT: Hierarchical Aggregation Transformers for Person Re-identification

Segmentation models with pretrained backbones. PyTorch.

joint detection and semantic segmentation, based on ultralytics/yolov5,

Elastic weight consolidation technique for incremental learning.

Official implementation of Self-supervised Image-to-text and Text-to-image Synthesis

Block Sparse movement pruning

A2LP for short, ECCV2020 spotlight, Investigating SSL principles for UDA problems

🔎 Super-scale your images and run experiments with Residual Dense and Adversarial Networks.

Qcover is an open source effort to help exploring combinatorial optimization problems in Noisy Intermediate-scale Quantum(NISQ) processor.

Portfolio Optimization and Quantitative Strategic Asset Allocation in Python

Chinese Mandarin tts text-to-speech 中文 (普通话) 语音 合成 , by fastspeech 2 , implemented in pytorch, using waveglow as vocoder,

Task-based end-to-end model learning in stochastic optimization

Chinese Mandarin tts text-to-speech 中文 (普通话) 语音合成 , by fastspeech 2 , implemented in pytorch, using waveglow as vocoder,