LPAQA: Language model Prompt And Query Archive

This repository contains data and code for the paper How Can We Know What Language Models Know?

Install

Our repository is based on LAMA. Please download our fork from here and follow the instructions to set up the environment and download pre-trained language models.

# clone LPAQA
git clone https://github.com/jzbjyb/LPAQA.git LPAQA
# clone LAMA
git clone https://github.com/jzbjyb/LAMA.git LAMA
pushd LAMA && git reset --hard b6b1885c64de5981f249a8b65de25cb0802b4bd5 && rm -rf .git && popd
mv LAMA/* LPAQA/ && rm -rf LAMA
# follow the instructions to install
cd LPAQA
./setup.sh

Retrieve factual knowledge from LMs

For example, to query the owner of MSN (Microsoft is the answer), you can either use manually created prompts (x is owned by y):

python lama/eval_generation.py --lm bert --t "MSN is owned by [MASK]."

or use LPAQA that ensembles a diversity of prompts:

# mined prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/mine
# paraphrased prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/paraphrase

Language model Prompt And Query Archive

Related tags

Overview

LPAQA: Language model Prompt And Query Archive

Install

Retrieve factual knowledge from LMs

Owner

Code for models used in Bashiri et al., "A Flow-based latent state generative model of neural population responses to natural images".

Multi-label Co-regularization for Semi-supervised Facial Action Unit Recognition (NeurIPS 2019)

Subpopulation detection in high-dimensional single-cell data

Python codes for Lite Audio-Visual Speech Enhancement.

Here is the implementation of our paper S2VC: A Framework for Any-to-Any Voice Conversion with Self-Supervised Pretrained Representations.

Human4D Dataset tools for processing and visualization

OpenL3: Open-source deep audio and image embeddings

A Simple LSTM-Based Solution for "Heartbeat Signal Classification and Prediction" in Tianchi

Predict multi paths to a moving person depending on his trajectory history.

Rethinking Transformer-based Set Prediction for Object Detection

Code for paper PairRE: Knowledge Graph Embeddings via Paired Relation Vectors.

Mmdetection3d Noted - MMDetection3D is an open source object detection toolbox based on PyTorch

Power Core Simulator!

This is an official implementation for "SimMIM: A Simple Framework for Masked Image Modeling".

The Adapter-Bot: All-In-One Controllable Conversational Model

Config files for my GitHub profile.

DockStream: A Docking Wrapper to Enhance De Novo Molecular Design

A Peer-to-peer Platform for Secure, Privacy-preserving, Decentralized Data Science

Official PyTorch implementation of "Edge Rewiring Goes Neural: Boosting Network Resilience via Policy Gradient".

Theory-inspired Parameter Control Benchmarks for Dynamic Algorithm Configuration