This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Overview

2022_the_annotated_transformer

Goal

This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Key points

  • We have re-factored Harvard NLP's Annotated Trasformer into a shell script version.

  • Dataset utilized Multi30K. (The dataset is small, so you can see the results quickly even on computers with low specifications.)

  • We provide the Colab version along with the shell script version, making it easy to modify the model and test the method.

    https://colab.research.google.com/drive/1SrRmC_Ti8IepeHFNBZBjNxl_wkTSJReC?usp=sharing

  • Loss Graph can be drawn.

  • BLEU Score can be measured.

file structure

├── models
│   ├── __init__.py
│   ├── blocks
│   │   ├── __init__.py
│   │   ├── decoder_layer.py
│   │   ├── encoder_layer.py
│   ├── embedding
│   │   ├── __init__.py
│   │   ├── positional_encoding.py
│   │   └── token_embedding.py
│   ├── layers
│   │   ├── __init__.py
│   │   ├── layer_norm.py
│   │   ├── multi_headed_attention.py
│   │   ├── position_wise_feed_forward.py
│   │   └── sublayer_connection.py
│   ├── model
│   │   ├── __init__.py
│   │   ├── decoder.py
│   │   ├── encoder_decoder.py
│   │   ├── encoder.py
│   │   ├── generator.py
│   └── util.py
├── result
│   ├── loss_graph.png
│   ├── train_loss.txt
│   └── valid_loss.txt
├── saved
├── utils
    ├── __init__.py
    ├── batch.py
    ├── batch_size_fn.py
    ├── bleu.py
    ├── data_loader.py
    ├── epoch_time.py
    ├── greedy_decode.py
    ├── label_smoothing.py
    ├── make_model.py
    ├── NoamOpt.py
    ├── run_epoch.py
    ├── simple_loss_compute.py
    └── tokenizer.py
├── README.md
├── test.py
├── train.py
├── config.py
├── data.py
└── graph.py

Training Result

Train Validation loss graph

image

Test set(unseen data) Translation Example

image

Test set(unseen data) BLEU Score Average: 35.870847920953594

Reference

https://nlp.seas.harvard.edu/2018/04/03/attention.html

https://jalammar.github.io/illustrated-transformer/

https://www.facebook.com/groups/TensorFlowKR/permalink/1618169785190740/

https://github.com/hyunwoongko/transformer

Owner
신재욱
신재욱
2022-bridge - Example code belonging to the Bridge pattern video

Let's Take The Bridge Pattern To The Next Level This video covers how the bridge

11 Jun 14, 2022
Exploit grafana Pre-Auth LFI

Grafana-LFI-8.x Exploit grafana Pre-Auth LFI How to use python3

2 Jul 25, 2022
Some Attacks of Exchange SSRF ProxyLogon&ProxyShell

Some Attacks of Exchange SSRF This project is heavily replicated in ProxyShell, NtlmRelayToEWS https://mp.weixin.qq.com/s/GFcEKA48bPWsezNdVcrWag Get 1

Jumbo 129 Dec 30, 2022
This is a multi-password‌ cracking tool that can help you hack facebook accounts very quickly

Pro_Crack Facebook Fast Cracking Tool This is a multi-password‌ cracking tool that can help you hack facebook accounts very quickly Installation On Te

•JINN• 1 Jan 16, 2022
XSS scanner in python

DeadXSS XSS scanner in python How to Download: Step 1: git clone https://github.com/Deadeye0x/DeadXSS.git Step 2: cd DeadXSS Step 3: python3 DeadXSS.p

2 Jul 17, 2022
Infection Monkey - An automated pentest tool

Infection Monkey Data center Security Testing Tool Welcome to the Infection Monkey! The Infection Monkey is an open source security tool for testing a

Guardicore Ltd. 6k Jan 09, 2023
Code to do NF in HDR,HEVC,HPL,MPL

Netflix-DL 6.0 |HDR-HEVC-MPL-HPL NOT Working| ! Buy working netflix cdm from [em

4 Dec 28, 2021
An All-In-One Pure Python PoC for CVE-2021-44228

Python Log4RCE An all-in-one pure Python3 PoC for CVE-2021-44228. Configure Replace the global variables at the top of the script to your configuratio

Alexandre Lavoie 178 Nov 09, 2022
Tinyman exploit finder - Tinyman exploit finder for python

tinyman_exploit_finder There was a big tinyman exploit. You can read about it he

fish.exe 9 Dec 27, 2022
This tool was created in order to automate some basic OSINT tasks for penetration testing assingments.

This tool was created in order to automate some basic OSINT tasks for penetration testing assingments. The main feature that I haven't seen much anywhere is the downloadd google dork function where t

Tobias 5 May 31, 2022
Ducky Script is the payload language of Hak5 gear.

Ducky Script is the payload language of Hak5 gear. Since its introduction with the USB Rubber Ducky in 2010, Ducky Script has grown in capability while maintaining simplicity. Aided by Bash for logic

Abir Abedin Khan 6 Oct 07, 2022
Hadoop Yan ResourceManager unauthorized RCE

Vuln Impact There was an unauthorized access vulnerability in Hadoop yarn ResourceManager. This vulnerability existed in Hadoop yarn, the core compone

Al1ex 25 Nov 24, 2022
Spring-0day/CVE-2022-22965

CVE-2022-22965 Spring Framework/CVE-2022-22965 Vulnerability ID: CVE-2022-22965/CNVD-2022-23942/QVD-2022-1691 Reproduce the vulnerability docker pull

iak 4 Apr 05, 2022
Discord-keylogger - Discord keylogger With Python

Discord-keylogger Usage python dlogger.py -t [Time interval in sec] if not speci

Satwik Sinha 1 Jan 30, 2022
A small Python Script To get all levels of subdomains from a list

getlevels A small Python Script To get all levels of subdomains Easily get 1st level, 2nd level, 3rd level, 4th level .... nth level subdomains Usag

9 Feb 15, 2022
Getting my gitlab commit history into github

🔰 ᵀᴱᴸᴱᴳᴿᴬᴹ ᴴᴬᶜᴷ ᴮᴼᵀ 🔰 The owner would not be responsible for any kind of bans due to the bot. • ⚡ INSTALLING ⚡ • • 🛠️ Lᴀɴɢᴜᴀɢᴇs Aɴᴅ Tᴏᴏʟs 🔰 • If

Santiago Chiesa 1 Dec 24, 2021
Guess the password for Tik Tok accounts

Guess the password for Tik Tok accounts Tool features : You don't need proxies There is no captcha Running on a private api Combo T

32 Dec 25, 2022
Worm/Trojan/Ransomware/apt/Rootkit/Virus Database

Pestilence - The Malware Database [] Screenshot Pestilence is a project created to make the possibility of malware analysis open and available to the

*ERR0R* 47 Dec 21, 2022
INFO 3350/6350, Spring 2022, Cornell

Information Science 3350/6350 Text mining for history and literature Staff and sections Instructor: Matthew Wilkens Graduate TAs: Federica Bologna, Ro

Wilkens Teaching 6 Feb 21, 2022
Cryptick is a stock ticker for cryptocurrency tokens, and a physical NFT.

Cryptick is a stock ticker for cryptocurrency tokens, and a physical NFT. This repository includes tools and documentation for the Cryptick device.

1 Dec 31, 2021