This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Overview

2022_the_annotated_transformer

Goal

This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Key points

  • We have re-factored Harvard NLP's Annotated Trasformer into a shell script version.

  • Dataset utilized Multi30K. (The dataset is small, so you can see the results quickly even on computers with low specifications.)

  • We provide the Colab version along with the shell script version, making it easy to modify the model and test the method.

    https://colab.research.google.com/drive/1SrRmC_Ti8IepeHFNBZBjNxl_wkTSJReC?usp=sharing

  • Loss Graph can be drawn.

  • BLEU Score can be measured.

file structure

├── models
│   ├── __init__.py
│   ├── blocks
│   │   ├── __init__.py
│   │   ├── decoder_layer.py
│   │   ├── encoder_layer.py
│   ├── embedding
│   │   ├── __init__.py
│   │   ├── positional_encoding.py
│   │   └── token_embedding.py
│   ├── layers
│   │   ├── __init__.py
│   │   ├── layer_norm.py
│   │   ├── multi_headed_attention.py
│   │   ├── position_wise_feed_forward.py
│   │   └── sublayer_connection.py
│   ├── model
│   │   ├── __init__.py
│   │   ├── decoder.py
│   │   ├── encoder_decoder.py
│   │   ├── encoder.py
│   │   ├── generator.py
│   └── util.py
├── result
│   ├── loss_graph.png
│   ├── train_loss.txt
│   └── valid_loss.txt
├── saved
├── utils
    ├── __init__.py
    ├── batch.py
    ├── batch_size_fn.py
    ├── bleu.py
    ├── data_loader.py
    ├── epoch_time.py
    ├── greedy_decode.py
    ├── label_smoothing.py
    ├── make_model.py
    ├── NoamOpt.py
    ├── run_epoch.py
    ├── simple_loss_compute.py
    └── tokenizer.py
├── README.md
├── test.py
├── train.py
├── config.py
├── data.py
└── graph.py

Training Result

Train Validation loss graph

image

Test set(unseen data) Translation Example

image

Test set(unseen data) BLEU Score Average: 35.870847920953594

Reference

https://nlp.seas.harvard.edu/2018/04/03/attention.html

https://jalammar.github.io/illustrated-transformer/

https://www.facebook.com/groups/TensorFlowKR/permalink/1618169785190740/

https://github.com/hyunwoongko/transformer

Owner
신재욱
신재욱
SARA - Simple Android Ransomware Attack

SARA - Simple Android Ransomware Attack Disclaimer The author is not responsible for any issues or damage caused by this program. Features User can cu

Termux Hackers 99 Jan 04, 2023
A bitcoin private keys brute-forcing tool. Educational purpose only.

BitForce A bitcoin private keys brute-forcing tool. If you have an average computer, his will take decades to find a private key with balance. Run Mak

Gilad Leef 2 Dec 20, 2022
Python exploit for vsftpd 2.3.4 - Backdoor Command Execution

CVE-2011-2523 - vsftpd 2.3.4 Exploit Discription vsftpd, which stands for Very Secure FTP Daemon,is an FTP server for Unix-like systems, including Lin

Padsala Tushal 5 Nov 08, 2022
SSLyze is a fast and powerful SSL/TLS scanning tool and Python library.

SSLyze SSLyze is a fast and powerful SSL/TLS scanning tool and Python library. SSLyze can analyze the SSL/TLS configuration of a server by connecting

Alban Diquet 2.8k Jan 03, 2023
SEBUAH TOOLS TERMUX CRACK AKUN FF HOMKI AKUN EPEP DAH SATU FOLLOW AE YA BROO AWOKWOK

print " INSTALL TOOLS " $ pkg update && upgrade $ pkg install python2 $ pkg install git $ pip2 install lolcat $ pip2 install bs4 $ pip2 install reques

Jeeck 2 Nov 29, 2021
Collection Of Discord Hacking Tools / Fun Stuff / Exploits That Is Completely Made Using Python.

Venom Collection Of Discord Hacking Tools / Fun Stuff / Exploits That Is Completely Made Using Python. Report Bug · Request Feature Contributing Well,

PndaBoi 25 Dec 06, 2022
Hack any account sending fake nitro QR code (only for educational purpose)

DISCORD_ACCOUNT_HACKING_TOOL ( EDUCATIONAL PURPOSE ) Hack any account sending fake nitro QR code (only for educational purpose) Start my program token

Novy 7 Jan 07, 2022
Fast subdomain scanner, Takes arguments from a Json file ("args.json") and outputs the subdomains.

Fast subdomain scanner, Takes arguments from a Json file ("args.json") and outputs the subdomains. File Structure core/ colors.py db/ wordlist.txt REA

whoami security 4 Jul 02, 2022
Static Token And Credential Scanner

Static Token And Credential Scanner What is it? STACS is a YARA powered static credential scanner which suports binary file formats, analysis of neste

STACS 81 Dec 27, 2022
A curated list of amazingly awesome Cybersecurity datasets

A curated list of amazingly awesome Cybersecurity datasets

758 Dec 28, 2022
Cve-2022-23131 - Cve-2022-23131 zabbix-saml-bypass-exp

cve-2022-23131 cve-2022-23131 zabbix-saml-bypass-exp replace [zbx_signed_session

东方有鱼名为咸 135 Dec 14, 2022
Encrypted Python Password Manager

PyPassKeep Encrypted Python Password Manager About PyPassKeep (PPK for short) is an encrypted python password manager used to secure your passwords fr

KrisIsHere 1 Nov 17, 2021
An intranet tool for easily intranet pentesting

IntarKnife v1.0 a tool can be used in intarnet for easily pentesting moudle hash spray U can use this tool to spray hash on a webshell IntraKnife.exe

4 Nov 24, 2021
Small Python library that adds password hashing methods to ORM objects

Password Mixin Mixin that adds some useful methods to ORM objects Compatible with Python 3.5 = 3.9 Install pip install password-mixin Setup first cre

Joe Gasewicz 5 Nov 22, 2022
GitLab CI security tools runner

Common Security Pipeline Описание проекта: Данный проект является вариантом реализации DevSecOps практик, на базе: GitLab DefectDojo OpenSouce tools g

Сити-Мобил 14 Dec 23, 2022
This tool allows to automatically test for Content Security Policy bypass payloads.

CSPass This tool allows to automatically test for Content Security Policy bypass payloads. Usage [cspass]$ ./cspass.py -h usage: cspass.py [-h] [--no-

Ruulian 30 Nov 22, 2022
Burp Suite extension for encoding/decoding EVM calldata

unblocker Burp Suite extension for encoding/decoding EVM calldata 0x00_prerequisites Burp Suite Java 8+ Python 2.7 0x01_installation clone this reposi

Halborn 16 Aug 30, 2022
WebLogic T3/IIOP RCE ExternalizableHelper.class of coherence.jar

CVE-2020-14756 WebLogic T3/IIOP RCE ExternalizableHelper.class of coherence.jar README project base on https://github.com/Y4er/CVE-2020-2555 and weblo

Y4er 77 Dec 06, 2022
A small Python Script To get all levels of subdomains from a list

getlevels A small Python Script To get all levels of subdomains Easily get 1st level, 2nd level, 3rd level, 4th level .... nth level subdomains Usag

9 Feb 15, 2022
LittleBrother is a simple parental control application monitoring specific processes on Linux hosts to monitor and limit the play time of children.

Parental Control Application LittleBrother Overview LittleBrother is a simple parental control application monitoring specific processes (read "games"

40 Dec 21, 2022