Utilizing RBERT model for KLUE Relation Extraction task

Last update: Nov 15, 2022

Related tags

Text Data & NLP KLUE-RBERT

Overview

RBERT for Relation Extraction task for KLUE

Project Description

Relation Extraction task is one of the task of Korean Language Understanding Evaluation(KLUE) Benchmark.
Relation extraction can be defined as multiclass classification task for relationship between subject entity and object entity.
Classes are such as no_relation, per:employee_of, org:founded_by... totaling 30 labels.
This repo contains custom fine-tuning method utilizing monologg's R-BERT Implementation.
Custom punctuations with Pororo NER has been added to the dataset prior to the model's training.
If you want to refer to the experimentation note such as punctuation method of the entity, please refer to the blog post

Arguments Usage

Argument	type	Default	Explanation
batch_size	int	40	batch size for training and inferece
num_folds	int	5	number of fold for Stratified KFold
num_train_epochs	int	5	number of epochs for training
loss	str	focalloss	loss function
gamma	float	1.0	focalloss's gamma value
optimizer	str	adamp	optimizer for training
scheduler	str	get_cosine_schedule_with_warmup	learning rate scheduler
learning_rate	float	0.00005	initial learning rate
weight_decay	float	0.01	Loss function's weight decay, preventing overfit
warmup_step	int	500
debug	bool	false	debug with CPU device for better error representation
dropout_rate	float	0.1
save_steps	int	100	number of steps for saving the model
evaluation_steps	int	100	number of step until the evaluation
metric_for_best_model	str	eval/loss	the metric for determining which is the best model
load_best_model_at_end	bool	True

References

Authorship

Hardware

GPU : Tesla V100 32GB

Owner

snoop2head

break, compose, display

snoop2head

GitHub Repository https://snoop2head.github.io/Relation-Extraction/

BERTAC (BERT-style transformer-based language model with Adversarially pretrained Convolutional neural network)

BERTAC (BERT-style transformer-based language model with Adversarially pretrained Convolutional neural network) BERTAC is a framework that combines a

6 Jan 24, 2022

KLUE-baseline contains the baseline code for the Korean Language Understanding Evaluation (KLUE) benchmark.

KLUE Baseline Korean(한국어) KLUE-baseline contains the baseline code for the Korean Language Understanding Evaluation (KLUE) benchmark. See our paper fo

74 Dec 13, 2022

:P Some basic stuff I'm gonna use for my upcoming Agile Software Development and Devops

reverse-image-search-py bash script.sh img_name.jpg Requirements pip install requests pip install pyshorteners Dry run [ Sudhanva M 3 Dec 18, 2021

An Explainable Leaderboard for NLP

ExplainaBoard: An Explainable Leaderboard for NLP Introduction | Website | Download | Backend | Paper | Video | Bib Introduction ExplainaBoard is an i

319 Dec 20, 2022

Simple virtual assistant using pyttsx3 and speech recognition optionally with pywhatkit and pther libraries.

VirtualAssistant Simple virtual assistant using pyttsx3 and speech recognition optionally with pywhatkit and pther libraries. Third Party Libraries us

1 Nov 27, 2021

A curated list of FOSS tools to improve the Hacker News experience

Awesome-Hackernews Hacker News is a social news website focusing on computer technologies, hacking and startups. It promotes any content likely to "gr

141 Dec 27, 2022

Share constant definitions between programming languages and make your constants constant again

Introduction Reconstant lets you share constant and enum definitions between programming languages. Constants are defined in a yaml file and converted

47 Sep 10, 2022

A PyTorch implementation of the Transformer model in "Attention is All You Need".

Attention is all you need: A Pytorch Implementation This is a PyTorch implementation of the Transformer model in "Attention is All You Need" (Ashish V

7.1k Jan 05, 2023

Kinky furry assitant based on GPT2

KinkyFurs-V0 Kinky furry assistant based on GPT2 How to run python3 V0.py then, open web browser and go to localhost:8080 Requirements: Flask trans

1 Jun 11, 2022

official ( API ) for the zAmericanEnglish app in [ Google play ] and [ App store ]

official ( API ) for the zAmericanEnglish app in [ Google play ] and [ App store ]

3 Jan 12, 2022

Stuff related to Ben Eater's 8bit breadboard computer

8bit breadboard computer simulator This is an assembler + simulator/emulator of Ben Eater's 8bit breadboard computer. For a version with its RAM upgra

29 Dec 29, 2022

FastFormers - highly efficient transformer models for NLU

FastFormers FastFormers provides a set of recipes and methods to achieve highly efficient inference of Transformer models for Natural Language Underst

678 Jan 05, 2023

Official Stanford NLP Python Library for Many Human Languages

Official Stanford NLP Python Library for Many Human Languages

6.4k Jan 02, 2023

Source code of the "Graph-Bert: Only Attention is Needed for Learning Graph Representations" paper

Graph-Bert Source code of "Graph-Bert: Only Attention is Needed for Learning Graph Representations". Please check the script.py as the entry point. We

14 Mar 25, 2022

Predict an emoji that is associated with a text

Sentiment Analysis Sentiment analysis in computational linguistics is a general term for techniques that quantify sentiment or mood in a text. Can you

30 Sep 07, 2022

PyTorch Implementation of the paper Single Image Texture Translation for Data Augmentation

SITT The repo contains official PyTorch Implementation of the paper Single Image Texture Translation for Data Augmentation. Authors: Boyi Li Yin Cui T

52 Jan 05, 2023

Code for the paper TestRank: Bringing Order into Unlabeled Test Instances for Deep Learning Tasks

TestRank in Pytorch Code for the paper TestRank: Bringing Order into Unlabeled Test Instances for Deep Learning Tasks by Yu Li, Min Li, Qiuxia Lai, Ya

3 May 19, 2022

Finds snippets in iambic pentameter in English-language text and tries to combine them to a rhyming sonnet.

Sonnet finder Finds snippets in iambic pentameter in English-language text and tries to combine them to a rhyming sonnet. Usage This is a Python scrip

11 Sep 25, 2022

Open Source Neural Machine Translation in PyTorch

OpenNMT-py: Open-Source Neural Machine Translation OpenNMT-py is the PyTorch version of the OpenNMT project, an open-source (MIT) neural machine trans

5.8k Jan 04, 2023

Random-Word-Generator - Generates meaningful words from dictionary with given no. of letters and words.

Random Word Generator Generates meaningful words from dictionary with given no. of letters and words. This might be useful for generating short links

1 Jan 01, 2022