An example project using OpenPrompt under pytorch-lightning for prompt-based SST2 sentiment analysis model

Last update: Oct 21, 2022

Related tags

Overview

pl_prompt_sst

An example project using OpenPrompt under the framework of pytorch-lightning for a training prompt-based text classification model on SST2 sentiment analysis dataset. Leveraging the pytorch-lightning features like logging, gradient accumulation and early stopping, etc. Can be used as a template for further development.

Run

Install requirement

pip install -r requirements.txt

Setup the prompt to use in sst2/prompt_config.json

{
    "template_text": "{\"placeholder\": \"text_a\"} In summary, the film was {\"mask\"}.",
    "label_words": [["bad"], ["good"]]
}

Adjust the arguments in run.sh or the code below for your need, and run it.

CUDA_VISIBLE_DEVICES=0 python -u main.py --input_dir ./sst2 \
                                         --prompt_config_dir ./sst2/prompt_config.json \
                                         --model_class bert \
                                         --model_name_or_path prajjwal1/bert-tiny \
                                         --lr 2e-4
                                         --bs 32 \
                                         --max_seq_length 64 \
                                         --patience 4 \
                                         --accumulation 2 \
                                         --seed 666

In my preliminary experiment with the settings above, the model achieve 0.822 F1 compared to 0.820 without prompt.

Note

Can only be executed after this fix on state_dict()

An example project using OpenPrompt under pytorch-lightning for prompt-based SST2 sentiment analysis model

Related tags

Overview

pl_prompt_sst

Run

Note

Owner

Zhiling Zhang

Yet another Python binding for fastText

Deploying a Text Summarization NLP use case on Docker Container Utilizing Nvidia GPU

Google AI 2018 BERT pytorch implementation

LSTC: Boosting Atomic Action Detection with Long-Short-Term Context

Residual2Vec: Debiasing graph embedding using random graphs

Create a semantic search engine with a neural network (i.e. BERT) whose knowledge base can be updated

A benchmark for evaluation and comparison of various NLP tasks in Persian language.

:house_with_garden: Fast & easy transfer learning for NLP. Harvesting language models for the industry. Focus on Question Answering.

Repo for Enhanced Seq2Seq Autoencoder via Contrastive Learning for Abstractive Text Summarization

PyTorch Implementation of the paper Single Image Texture Translation for Data Augmentation

Syntax-aware Multi-spans Generation for Reading Comprehension (TASLP 2022)

HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

Blazing fast language detection using fastText model

Source code for AAAI20 "Generating Persona Consistent Dialogues by Exploiting Natural Language Inference".

All the code I wrote for Overwatch-related projects that I still own the rights to.

This is the Alpha of Nutte language, she is not complete yet / Essa é a Alpha da Nutte language, não está completa ainda

Learn meanings behind words is a key element in NLP. This project concentrates on the disambiguation of preposition senses. Therefore, we train a bert-transformer model and surpass the state-of-the-art.

中文問句產生器；使用台達電閱讀理解資料集(DRCD)

Tevatron is a simple and efficient toolkit for training and running dense retrievers with deep language models.

Pretrained Japanese BERT models