This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Overview

2022_the_annotated_transformer

Goal

This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Key points

  • We have re-factored Harvard NLP's Annotated Trasformer into a shell script version.

  • Dataset utilized Multi30K. (The dataset is small, so you can see the results quickly even on computers with low specifications.)

  • We provide the Colab version along with the shell script version, making it easy to modify the model and test the method.

    https://colab.research.google.com/drive/1SrRmC_Ti8IepeHFNBZBjNxl_wkTSJReC?usp=sharing

  • Loss Graph can be drawn.

  • BLEU Score can be measured.

file structure

├── models
│   ├── __init__.py
│   ├── blocks
│   │   ├── __init__.py
│   │   ├── decoder_layer.py
│   │   ├── encoder_layer.py
│   ├── embedding
│   │   ├── __init__.py
│   │   ├── positional_encoding.py
│   │   └── token_embedding.py
│   ├── layers
│   │   ├── __init__.py
│   │   ├── layer_norm.py
│   │   ├── multi_headed_attention.py
│   │   ├── position_wise_feed_forward.py
│   │   └── sublayer_connection.py
│   ├── model
│   │   ├── __init__.py
│   │   ├── decoder.py
│   │   ├── encoder_decoder.py
│   │   ├── encoder.py
│   │   ├── generator.py
│   └── util.py
├── result
│   ├── loss_graph.png
│   ├── train_loss.txt
│   └── valid_loss.txt
├── saved
├── utils
    ├── __init__.py
    ├── batch.py
    ├── batch_size_fn.py
    ├── bleu.py
    ├── data_loader.py
    ├── epoch_time.py
    ├── greedy_decode.py
    ├── label_smoothing.py
    ├── make_model.py
    ├── NoamOpt.py
    ├── run_epoch.py
    ├── simple_loss_compute.py
    └── tokenizer.py
├── README.md
├── test.py
├── train.py
├── config.py
├── data.py
└── graph.py

Training Result

Train Validation loss graph

image

Test set(unseen data) Translation Example

image

Test set(unseen data) BLEU Score Average: 35.870847920953594

Reference

https://nlp.seas.harvard.edu/2018/04/03/attention.html

https://jalammar.github.io/illustrated-transformer/

https://www.facebook.com/groups/TensorFlowKR/permalink/1618169785190740/

https://github.com/hyunwoongko/transformer

Owner
신재욱
신재욱
Detection And Breaking With Python

Detection And Breaking IIIIIIIIIIIIIIIIIIII PPPPPPPPPPPPPPPPP VVVVVVVV VVVVVVVV I::::::::II::::::::I P:::::::

Baris Dincer 1 Dec 26, 2021
大宝剑-信息收集和资产梳理工具(红队、蓝队、企业组织架构、子域名、Web资产梳理、Web指纹识别、ICON_Hash资产匹配)

大宝剑-信息收集和资产梳理工具(红队、蓝队、企业组织架构、子域名、Web资产梳理、Web指纹识别、ICON_Hash资产匹配)

Wolf Group Security Team 835 Jan 05, 2023
🎻 Modularized exploit generation framework

Modularized exploit generation framework for x86_64 binaries Overview This project is still at early stage of development, so you might want to come b

ᴀᴇꜱᴏᴘʜᴏʀ 30 Jan 17, 2022
⛤Keylogger Generator for Windows written in Python⛤

⛤Keylogger Generator for Windows written in Python⛤

FZGbzuw412 33 Nov 24, 2022
Script Crack Facebook Premium 🚶‍♂

premium Script Crack Facebook Premium 🚶‍♂ In Script Install Script $ pkg update && pkg upgrade $ termux-setup-storage $ pkg install python $ pkg inst

Yumasaa 2 Dec 19, 2021
log4j2 dos exploit,CVE-2021-45105 exploit,Denial of Service poc

说明 about author: 我超怕的 blog: https://www.cnblogs.com/iAmSoScArEd/ github: https://github.com/iAmSOScArEd/ date: 2021-12-20 log4j2 dos exploit log4j2 do

3 Aug 13, 2022
Instagram brute force tool that uses tor as its proxy connections

Insta-crack This is a instagram brute force tool that uses tor as its proxy connections, keep in mind that you should not do anything illegal with thi

Liam 3 Jan 28, 2022
CVE-2021-26855 SSRF Exchange Server

CVE-2021-26855 Brute Force EMail Exchange Server Timeline: Monday, March 8, 2021: Update Dumping content...(I'm not done, can u guy help me done this

lulz 117 Nov 28, 2022
A Proof-Of-Concept for the recently found CVE-2021-44228 vulnerability

log4j-shell-poc A Proof-Of-Concept for the recently found CVE-2021-44228 vulnerability. Recently there was a new vulnerability in log4j, a java loggin

koz 1.5k Jan 04, 2023
Virus-Builder - This tool will generate a virus that can only destroy Windows computer

Virus-Builder - This tool will generate a virus that can only destroy Windows computer. You can also configure to auto run in usb drive

Saad 16 Dec 30, 2022
Microsoft Exchange Server SSRF漏洞(CVE-2021-26855)

Microsoft_Exchange_Server_SSRF_CVE-2021-26855 zoomeye dork:app:"Microsoft Exchange Server" 使用Seebug工具箱及pocsuite3编写的脚本Microsoft_Exchange_Server_SSRF_CV

conjojo 37 Nov 12, 2022
A high-performance DNS stub resolver for bulk lookups and reconnaissance (subdomain enumeration)

MassDNS A high-performance DNS stub resolver MassDNS is a simple high-performance DNS stub resolver targeting those who seek to resolve a massive amou

B. Blechschmidt 2.5k Jan 07, 2023
A passive-recon tool that parses through found assets and interacts with the Hackerone API

Hackerone Passive Recon Tool A passive-recon tool that parses through found assets and interacts with the Hackerone API. Setup Simply run setup.sh to

elbee 4 Jan 13, 2022
Windows Virus who destroy some impotants files on C:\windows\system32\

psychic-robot Windows Virus who destroy some importants files on C:\windows\system32\ Signatures of psychic-robot.PY (python file) : Bkav Pro : ASP.We

H-Tech-Dev36 1 Jan 06, 2022
A python implementation of the windows 95 product key check.

Windows 95 Product Key Check Info: This is a python implementation of the windows 95 product key check. This was just a bit of fun and a massive 5 hou

11 Aug 07, 2022
These are Simple python scripts to test/scan your network

Disclaimer This tool is for Educational purpose only. We do not promote or encourage any illegal activities. Summary These are Simple python scripts t

Varun Jagtap 5 Oct 08, 2022
Simple script to have LDAP authentication in Home Assistant Docker, using NGINX's ldap-auth container

Home Assistant LDAP Auth Simple script to have LDAP authentication in Home Assistant Docker, using NGINX's ldap-auth container. Usage Deploy NGINX's l

Erik 1 Sep 21, 2022
Log4j vuln fuzz/scan with python

Log4jFuzz log4j vuln fuzz/scan USE // it's use localhost udp server to check target vuln. python3 log4jFuzz.py [option] optional arguments: -u URL,

VVzv 3 Dec 22, 2021
This repo is about steps to create a effective custom wordlist in a few clicks/

Custom Wordlist This repo is about steps to take in order to create a effective custom wordlist in a few clicks. this comes handing in pentesting enga

2 Oct 08, 2022
Abusing Microsoft 365 OAuth Authorization Flow for Phishing Attack

O365DevicePhish Microsoft365_devicePhish Abusing Microsoft 365 OAuth Authorization Flow for Phishing Attack This is a simple proof-of-concept script t

Trewis [work] Scotch 4 Sep 23, 2022