LPAQA: Language model Prompt And Query Archive

This repository contains data and code for the paper How Can We Know What Language Models Know?

Install

Our repository is based on LAMA. Please download our fork from here and follow the instructions to set up the environment and download pre-trained language models.

# clone LPAQA
git clone https://github.com/jzbjyb/LPAQA.git LPAQA
# clone LAMA
git clone https://github.com/jzbjyb/LAMA.git LAMA
pushd LAMA && git reset --hard b6b1885c64de5981f249a8b65de25cb0802b4bd5 && rm -rf .git && popd
mv LAMA/* LPAQA/ && rm -rf LAMA
# follow the instructions to install
cd LPAQA
./setup.sh

Retrieve factual knowledge from LMs

For example, to query the owner of MSN (Microsoft is the answer), you can either use manually created prompts (x is owned by y):

python lama/eval_generation.py --lm bert --t "MSN is owned by [MASK]."

or use LPAQA that ensembles a diversity of prompts:

# mined prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/mine
# paraphrased prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/paraphrase

Language model Prompt And Query Archive

Related tags

Overview

LPAQA: Language model Prompt And Query Archive

Install

Retrieve factual knowledge from LMs

Owner

Code for the paper "Learning-Augmented Algorithms for Online Steiner Tree"

The versatile ocean simulator, in pure Python, powered by JAX.

(NeurIPS 2020) Wasserstein Distances for Stereo Disparity Estimation

Official repository of ICCV21 paper "Viewpoint Invariant Dense Matching for Visual Geolocalization"

Activity tragle - Google is tracking everything, we just look at it

Pytorch implementation of various High Dynamic Range (HDR) Imaging algorithms

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors, CVPR 2021

Keras + Hyperopt: A very simple wrapper for convenient hyperparameter optimization

Official implementation of the NRNS paper: No RL, No Simulation: Learning to Navigate without Navigating

Learning To Have An Ear For Face Super-Resolution

Memory efficient transducer loss computation

Official Chainer implementation of GP-GAN: Towards Realistic High-Resolution Image Blending (ACMMM 2019, oral)

Official implementation of SIGIR'2021 paper: "Sequential Recommendation with Graph Neural Networks".

[MICCAI'20] AlignShift: Bridging the Gap of Imaging Thickness in 3D Anisotropic Volumes

Optimus: the first large-scale pre-trained VAE language model

CSAC - Collaborative Semantic Aggregation and Calibration for Separated Domain Generalization

Differential Privacy for Heterogeneous Federated Learning : Utility & Privacy tradeoffs

A library built upon PyTorch for building embeddings on discrete event sequences using self-supervision

Change Detection in SAR Images Based on Multiscale Capsule Network

A python3 tool to take a 360 degree survey of the RF spectrum (hamlib + rotctld + RTL-SDR/HackRF)