This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Overview

2022_the_annotated_transformer

Goal

This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Key points

  • We have re-factored Harvard NLP's Annotated Trasformer into a shell script version.

  • Dataset utilized Multi30K. (The dataset is small, so you can see the results quickly even on computers with low specifications.)

  • We provide the Colab version along with the shell script version, making it easy to modify the model and test the method.

    https://colab.research.google.com/drive/1SrRmC_Ti8IepeHFNBZBjNxl_wkTSJReC?usp=sharing

  • Loss Graph can be drawn.

  • BLEU Score can be measured.

file structure

├── models
│   ├── __init__.py
│   ├── blocks
│   │   ├── __init__.py
│   │   ├── decoder_layer.py
│   │   ├── encoder_layer.py
│   ├── embedding
│   │   ├── __init__.py
│   │   ├── positional_encoding.py
│   │   └── token_embedding.py
│   ├── layers
│   │   ├── __init__.py
│   │   ├── layer_norm.py
│   │   ├── multi_headed_attention.py
│   │   ├── position_wise_feed_forward.py
│   │   └── sublayer_connection.py
│   ├── model
│   │   ├── __init__.py
│   │   ├── decoder.py
│   │   ├── encoder_decoder.py
│   │   ├── encoder.py
│   │   ├── generator.py
│   └── util.py
├── result
│   ├── loss_graph.png
│   ├── train_loss.txt
│   └── valid_loss.txt
├── saved
├── utils
    ├── __init__.py
    ├── batch.py
    ├── batch_size_fn.py
    ├── bleu.py
    ├── data_loader.py
    ├── epoch_time.py
    ├── greedy_decode.py
    ├── label_smoothing.py
    ├── make_model.py
    ├── NoamOpt.py
    ├── run_epoch.py
    ├── simple_loss_compute.py
    └── tokenizer.py
├── README.md
├── test.py
├── train.py
├── config.py
├── data.py
└── graph.py

Training Result

Train Validation loss graph

image

Test set(unseen data) Translation Example

image

Test set(unseen data) BLEU Score Average: 35.870847920953594

Reference

https://nlp.seas.harvard.edu/2018/04/03/attention.html

https://jalammar.github.io/illustrated-transformer/

https://www.facebook.com/groups/TensorFlowKR/permalink/1618169785190740/

https://github.com/hyunwoongko/transformer

Owner
신재욱
신재욱
Generate malicious files using recently published homoglyphic-attack (CVE-2021-42694)

CVE-2021-42694 Generate malicious files using recently published homoglyph-attack vulnerability, which was discovered at least in C, C++, C#, Go, Pyth

js-on 17 Dec 11, 2022
Getting my gitlab commit history into github

🔰 ᵀᴱᴸᴱᴳᴿᴬᴹ ᴴᴬᶜᴷ ᴮᴼᵀ 🔰 The owner would not be responsible for any kind of bans due to the bot. • ⚡ INSTALLING ⚡ • • 🛠️ Lᴀɴɢᴜᴀɢᴇs Aɴᴅ Tᴏᴏʟs 🔰 • If

Santiago Chiesa 1 Dec 24, 2021
🏃 Python Solutions of All Problems in FHC 2021 (In Progress)

FacebookHackerCup-2021 Python solutions of Facebook Hacker Cup 2021. Solution begins with * means it will get TLE in the largest data set (total compu

kamyu 14 Oct 15, 2022
Delta Sharing: An Open Protocol for Secure Data Sharing

Delta Sharing: An Open Protocol for Secure Data Sharing Delta Sharing is an open protocol for secure real-time exchange of large datasets, which enabl

Delta Lake 497 Jan 02, 2023
Python APK Reverser & Patcher Tool

DTL-X An Advanced Python APK Reverser and Patcher Tool. --rmads1: target=AndroidManifest.xml,replace=com.google.android.gms.ad --rmads2: No Internet (

DedSecTL 10 Oct 31, 2022
Scout Suite - an open source multi-cloud security-auditing tool,

Description Scout Suite is an open source multi-cloud security-auditing tool, which enables security posture assessment of cloud environments. Using t

NCC Group Plc 5k Jan 05, 2023
Automatic ProxyShell Exploit

proxyshell-auto usage: proxyshell.py [-h] -t T Automatic Exploit ProxyShell optional arguments: -h, --help show this help message and exit -t T

lulz 93 Jan 05, 2023
Yet another web fuzzer

yafuzz Yet another web fuzzer Usage This script can run in two modes of operation. Supplying a wordlist -W argument will initiate a multithreaded fuzz

FooBallZ 5 Feb 02, 2022
Provides script to download and format public IP lists related to the Log4j exploit.

Provides script to download and format public IP lists related to the Log4j exploit. Current format includes: plain list, Cisco ASA Network Group.

Gianluca Ulivi 1 Jan 02, 2022
labsecurity is a tool that brings together python scripts made for ethical hacking, in a single tool, through a console interface

labsecurity labsecurity is a tool that brings together python scripts made for ethical hacking, in a single tool, through a console interface. Warning

Dylan Meca 16 Dec 08, 2022
Aiminsun 165 Dec 21, 2022
Check for breached passwords with k-anonymity

passwnd Check for breached passwords with k-anonymity Usage To get prompted to enter the password securely, simply run: passwnd.py Alternatively, you

Nat 1 Feb 08, 2022
Log4j minecraft with python

log4jminecraft This code DOES NOT promote or encourage any illegal activities! The content in this document is provided solely for educational purpose

David Bombal 154 Dec 24, 2022
Statistical Random Number Generator Attack Against The Kirchhoff-law-johnson-noise (Kljn) Secure Key Exchange Protocol

Statistical Random Number Generator Attack Against The Kirchhoff-law-johnson-noise (Kljn) Secure Key Exchange Protocol

zeze 1 Jan 13, 2022
Worm/Trojan/Ransomware/apt/Rootkit/Virus Database

Pestilence - The Malware Database [] Screenshot Pestilence is a project created to make the possibility of malware analysis open and available to the

*ERR0R* 47 Dec 21, 2022
Dapunta Multi Brute Force Facebook - Crack Facebook With Login - Free

✭ DMBF CRACK Dibuat Dengan ❤️ Oleh Dapunta Author: - Dapunta Khurayra X ⇨ Fitur Login [✯] Login Token ⇨ Fitur Crack [✯] Crack Dari Teman, Public,

Dapunta ID 10 Oct 19, 2022
Simple Python 3 script to detect the "Log4j" Java library vulnerability (CVE-2021-44228) for a list of URL with multithreading

log4j-detect Simple Python 3 script to detect the "Log4j" Java library vulnerability (CVE-2021-44228) for a list of URL with multithreading The script

Wade 1 Dec 15, 2021
Trainspotting - Python Dependency Injector based on interface binding

Choose dependency injection Friendly with MyPy Supports lazy injections Supports

avito.tech 3 Jan 26, 2022
Webpack自动化信息收集

Webpack-信息收集工具 郑重声明:文中所涉及的技术、思路和工具仅供以安全为目的的学习交流使用,任何人不得将其用于非法用途以及盈利等目的,否则后果自行承担。 0x01 介绍 作者:小洲 团队:横戈安全团队,未来一段时间将陆续开源工具,欢迎关注微信公众号: 定位:协助红队人员快速的信息收集,测绘目

小洲 214 Dec 19, 2022
Something I built to test for Log4J vulnerabilities on customer networks.

Log4J-Scanner Something I built to test for Log4J vulnerabilities on customer networks. I'm not responsible if your computer blows up, catches fire or

1 Dec 20, 2021