Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Last update: Jan 08, 2023

Related tags

Overview

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Results on STS Tasks

Model	STS12	STS13	STS14	STS15	STS16	STSb	SICK-R	Avg.
unsup-prompt-bert-base Download	71.98	84.66	77.13	84.52	81.10	82.03	70.64	78.87
unsup-prompt-roberta-base Download	73.98	84.73	77.88	84.93	81.89	82.74	69.21	79.34
sup-prompt-bert-base Download	75.48	85.59	80.57	85.99	81.08	84.56	80.52	81.97
sup-prompt-roberta-base Download	76.75	85.93	82.28	86.69	82.80	86.14	80.04	82.95

Download Data

cd SentEval/data/downstream/
bash download_dataset.sh
cd -
cd ./data
bash download_wiki.sh
bash download_nli.sh
cd -

Static token embedding with removing embedding biases

robert-base, bert-base-cased and robert-base-uncased

./run.sh roberta-base-embedding-only-remove-baises
./run.sh bert-base-cased-embedding-only-remove-baises
./run.sh bert-base-uncased-embedding-only-remove-baises

Non fine-tuned BERT with Prompt

bert-base-uncased with prompt

./run.sh bert-prompt

bert-base-uncased with optiprompt

./run.sh bert-optiprompt

fine-tuned BERT with Prompt

unsupervised

SEED=0
./run.sh unsup-roberta $SEED

SEED=0
./run.sh unsup-bert $SEED

supervised

./run.sh sup-roberta

./run.sh sup-bert

Our Code is based on SimCSE

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Related tags

Overview

Prompt-BERT: Prompt makes BERT Better at Sentence Embeddings

Results on STS Tasks

Download Data

Static token embedding with removing embedding biases

Non fine-tuned BERT with Prompt

fine-tuned BERT with Prompt

unsupervised

supervised

Owner

HyDiff: Hybrid Differential Software Analysis

Social Network Ads Prediction

[CVPR 2021] Involution: Inverting the Inherence of Convolution for Visual Recognition, a brand new neural operator

Cross-Modal Contrastive Learning for Text-to-Image Generation

VM3000 Microphones

All supplementary material used by me while TA-ing CS3244: Machine Learning

Discovering and Achieving Goals via World Models

Automatically erase objects in the video, such as logo, text, etc.

A smaller subset of 10 easily classified classes from Imagenet, and a little more French

Home repository for the Regularized Greedy Forest (RGF) library. It includes original implementation from the paper and multithreaded one written in C++, along with various language-specific wrappers.

Data-depth-inference - Data depth inference with python

一些经典的CTR算法的复现; LR, FM, FFM, AFM, DeepFM，xDeepFM, PNN, DCN, DCNv2, DIFM, AutoInt, FiBiNet,AFN,ONN,DIN, DIEN ... （pytorch, tf2.0）

pytorch implementation of fast-neural-style

YOLOv7 - Framework Beyond Detection

Super-BPD: Super Boundary-to-Pixel Direction for Fast Image Segmentation (CVPR 2020)

TGRNet: A Table Graph Reconstruction Network for Table Structure Recognition

Out-of-distribution detection using the pNML regret. NeurIPS2021

Geometric Vector Perceptron --- a rotation-equivariant GNN for learning from biomolecular structure

Chinese license plate recognition

Dynamic View Synthesis from Dynamic Monocular Video