An implementation of the "Attention is all you need" paper without extra bells and whistles, or difficult syntax

Last update: Jun 16, 2022

Overview

Simple Transformer

An implementation of the "Attention is all you need" paper without extra bells and whistles, or difficult syntax.

Note: The only extra thing added is Dropout regularization in some layers and option to use GPU.

Install

python -m pip install -r requirements.txt

Toy data

python train_toy_data.py

Before Training	After 100 Epoch

English -> German Europarl dataset

python train_translate.py

Training on a small subset of 1000 sentences (Included in this repo)

Owner

GitHub Repository

Data Augmentation Using Keras and Python

Data-Augmentation-Using-Keras-and-Python Data augmentation is the process of increasing the number of training dataset. Keras library offers a simple

3 Feb 15, 2022

Code for "Training Neural Networks with Fixed Sparse Masks" (NeurIPS 2021).

Fisher Induced Sparse uncHanging (FISH) Mask This repo contains the code for Fisher Induced Sparse uncHanging (FISH) Mask training, from "Training Neu

37 Dec 30, 2022

The code for our paper "NSP-BERT: A Prompt-based Zero-Shot Learner Through an Original Pre-training Task —— Next Sentence Prediction"

201 Nov 21, 2022

Development kit for MIT Scene Parsing Benchmark

Development Kit for MIT Scene Parsing Benchmark [NEW!] Our PyTorch implementation is released in the following repository: https://github.com/hangzhao

424 Dec 01, 2022

Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation (CVPR 2021)

Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation Input Image Initial CAM Successive Maps with adversar

110 Dec 07, 2022

Comp445 project - Data Communications & Computer Networks

COMP-445 Data Communications & Computer Networks Change Python version in Conda

2 Oct 03, 2022

Multi-Modal Fingerprint Presentation Attack Detection: Evaluation On A New Dataset

PADISI USC Dataset This repository analyzes the PADISI-Finger dataset introduced in Multi-Modal Fingerprint Presentation Attack Detection: Evaluation

6 Feb 06, 2022

Equivariant layers for RC-complement symmetry in DNA sequence data

Equi-RC Equivariant layers for RC-complement symmetry in DNA sequence data This is a repository that implements the layers as described in "Reverse-Co

7 May 19, 2022

code for paper "Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning" by Zhongzheng Ren, Raymond A. Yeh, Alexander G. Schwing.

Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning Overview This code is for paper: Not All Unlabeled Data are Equa

22 Nov 23, 2022

Face Recognition & AI Based Smart Attendance Monitoring System.

In today’s generation, authentication is one of the biggest problems in our society. So, one of the most known techniques used for authentication is h

1 Jan 14, 2022

Hierarchical probabilistic 3D U-Net, with attention mechanisms (—𝘈𝘵𝘵𝘦𝘯𝘵𝘪𝘰𝘯 𝘜-𝘕𝘦𝘵, 𝘚𝘌𝘙𝘦𝘴𝘕𝘦𝘵) and a nested decoder structure with deep supervision (—𝘜𝘕𝘦𝘵++).

Hierarchical probabilistic 3D U-Net, with attention mechanisms (—𝘈𝘵𝘵𝘦𝘯𝘵𝘪𝘰𝘯 𝘜-𝘕𝘦𝘵, 𝘚𝘌𝘙𝘦𝘴𝘕𝘦𝘵) and a nested decoder structure with deep supervision (—𝘜𝘕𝘦𝘵++). Built in TensorFlow 2.5. Configured for vox

32 Dec 08, 2022

An implementation of the "Attention is all you need" paper without extra bells and whistles, or difficult syntax

Related tags

Overview

Simple Transformer

Install

Toy data

English -> German Europarl dataset

Owner

Data Augmentation Using Keras and Python

Code for "Training Neural Networks with Fixed Sparse Masks" (NeurIPS 2021).

The code for our paper "NSP-BERT: A Prompt-based Zero-Shot Learner Through an Original Pre-training Task —— Next Sentence Prediction"

Development kit for MIT Scene Parsing Benchmark

Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation (CVPR 2021)

Comp445 project - Data Communications & Computer Networks

Multi-Modal Fingerprint Presentation Attack Detection: Evaluation On A New Dataset

Equivariant layers for RC-complement symmetry in DNA sequence data

code for paper "Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning" by Zhongzheng Ren, Raymond A. Yeh, Alexander G. Schwing.

Face Recognition & AI Based Smart Attendance Monitoring System.

Hierarchical probabilistic 3D U-Net, with attention mechanisms (—𝘈𝘵𝘵𝘦𝘯𝘵𝘪𝘰𝘯 𝘜-𝘕𝘦𝘵, 𝘚𝘌𝘙𝘦𝘴𝘕𝘦𝘵) and a nested decoder structure with deep supervision (—𝘜𝘕𝘦𝘵++).

An Implementation of Fully Convolutional Networks in Tensorflow.

Official repository for ABC-GAN

Source code for EquiDock: Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Docking (ICLR 2022)

Learning Dynamic Network Using a Reuse Gate Function in Semi-supervised Video Object Segmentation.

Mixed Transformer UNet for Medical Image Segmentation

Garbage Detection system which will detect objects based on whether it is plastic waste or plastics or just garbage.

TensorLight - A high-level framework for TensorFlow

Video Swin Transformer - PyTorch

Official PyTorch implementation of "ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows"

An implementation of the "Attention is all you need" paper without extra bells and whistles, or difficult syntax

Related tags

Overview

Simple Transformer

Install

Toy data

English -> German Europarl dataset

Owner

Data Augmentation Using Keras and Python

Code for "Training Neural Networks with Fixed Sparse Masks" (NeurIPS 2021).

The code for our paper "NSP-BERT: A Prompt-based Zero-Shot Learner Through an Original Pre-training Task —— Next Sentence Prediction"

Development kit for MIT Scene Parsing Benchmark

Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation (CVPR 2021)

Comp445 project - Data Communications & Computer Networks

Multi-Modal Fingerprint Presentation Attack Detection: Evaluation On A New Dataset

Equivariant layers for RC-complement symmetry in DNA sequence data

code for paper "Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning" by Zhongzheng Ren*, Raymond A. Yeh*, Alexander G. Schwing.

Face Recognition & AI Based Smart Attendance Monitoring System.

Hierarchical probabilistic 3D U-Net, with attention mechanisms (—𝘈𝘵𝘵𝘦𝘯𝘵𝘪𝘰𝘯 𝘜-𝘕𝘦𝘵, 𝘚𝘌𝘙𝘦𝘴𝘕𝘦𝘵) and a nested decoder structure with deep supervision (—𝘜𝘕𝘦𝘵++).

An Implementation of Fully Convolutional Networks in Tensorflow.

Official repository for ABC-GAN

Source code for EquiDock: Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Docking (ICLR 2022)

Learning Dynamic Network Using a Reuse Gate Function in Semi-supervised Video Object Segmentation.

Mixed Transformer UNet for Medical Image Segmentation

Garbage Detection system which will detect objects based on whether it is plastic waste or plastics or just garbage.

TensorLight - A high-level framework for TensorFlow

Video Swin Transformer - PyTorch

Official PyTorch implementation of "ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows"

code for paper "Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning" by Zhongzheng Ren, Raymond A. Yeh, Alexander G. Schwing.