LPAQA: Language model Prompt And Query Archive

This repository contains data and code for the paper How Can We Know What Language Models Know?

Install

Our repository is based on LAMA. Please download our fork from here and follow the instructions to set up the environment and download pre-trained language models.

# clone LPAQA
git clone https://github.com/jzbjyb/LPAQA.git LPAQA
# clone LAMA
git clone https://github.com/jzbjyb/LAMA.git LAMA
pushd LAMA && git reset --hard b6b1885c64de5981f249a8b65de25cb0802b4bd5 && rm -rf .git && popd
mv LAMA/* LPAQA/ && rm -rf LAMA
# follow the instructions to install
cd LPAQA
./setup.sh

Retrieve factual knowledge from LMs

For example, to query the owner of MSN (Microsoft is the answer), you can either use manually created prompts (x is owned by y):

python lama/eval_generation.py --lm bert --t "MSN is owned by [MASK]."

or use LPAQA that ensembles a diversity of prompts:

# mined prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/mine
# paraphrased prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/paraphrase

Language model Prompt And Query Archive

Related tags

Overview

LPAQA: Language model Prompt And Query Archive

Install

Retrieve factual knowledge from LMs

Owner

Tensorflow/Keras Plug-N-Play Deep Learning Models Compilation

Code for Contrastive-Geometry Networks for Generalized 3D Pose Transfer

2021-AIAC-QQ-Browser-Hyperparameter-Optimization-Rank6

NitroFE is a Python feature engineering engine which provides a variety of modules designed to internally save past dependent values for providing continuous calculation.

Piotr - IoT firmware emulation instrumentation for training and research

PRIN/SPRIN: On Extracting Point-wise Rotation Invariant Features

Graph Neural Networks with Keras and Tensorflow 2.

The FIRST GANs-based omics-to-omics translation framework

Collection of common code that's shared among different research projects in FAIR computer vision team.

Deep Learning for Time Series Forecasting.

classification task on dataset-CIFAR10,by using Tensorflow/keras

[PAMI 2020] Show, Match and Segment: Joint Weakly Supervised Learning of Semantic Matching and Object Co-segmentation

A tensorflow/keras implementation of StyleGAN to generate images of new Pokemon.

Multi-Stage Episodic Control for Strategic Exploration in Text Games

Solver for Large-Scale Rank-One Semidefinite Relaxations

[ICLR 2021] "Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective" by Wuyang Chen, Xinyu Gong, Zhangyang Wang

This implements one of result networks from Large-scale evolution of image classifiers

A working implementation of the Categorical DQN (Distributional RL).

This is a Python Module For Encryption, Hashing And Other stuff

The author's officially unofficial PyTorch BigGAN implementation.