PyTorch Implementation for AAAI'21 "Do Response Selection Models Really Know What's Next? Utterance Manipulation Strategies for Multi-turn Response Selection"

Last update: Nov 22, 2022

Related tags

Deep Learning UMS-ResSel

Overview

UMS for Multi-turn Response Selection

Implements the model described in the following paper Do Response Selection Models Really Know What's Next? Utterance Manipulation Strategies for Multi-turn Response Selection.

@inproceedings{whang2021ums,
  title={Do Response Selection Models Really Know What's Next? Utterance Manipulation Strategies for Multi-turn Response Selection},
  author={Whang, Taesun and Lee, Dongyub and Oh, Dongsuk and Lee, Chanhee and Han, Kijong and Lee, Dong-hun and Lee, Saebyeok},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  year={2021}
}

This code is reimplemented as a fork of huggingface/transformers and taesunwhang/BERT-ResSel.

Setup and Dependencies

This code is implemented using PyTorch v1.6.0, and provides out of the box support with CUDA 10.1 and CuDNN 7.6.5.

Anaconda / Miniconda is the recommended to set up this codebase.

Anaconda or Miniconda

Clone this repository and create an environment:

git clone https://www.github.com/taesunwhang/UMS-ResSel
conda create -n ums_ressel python=3.7

# activate the environment and install all dependencies
conda activate ums_ressel
cd UMS-ResSel

# https://pytorch.org
pip install torch==1.6.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html
pip install -r requirements.txt

Preparing Data and Checkpoints

Pre- and Post-trained Checkpoints

We provide following pre- and post-trained checkpoints.

bert-base (english), bert-base-wwm (chinese)
bert-post (ubuntu, douban, e-commerce)
electra-base (english), electra-base (chinese)
electra-post (ubuntu, douban, e-commerce)

sh scripts/download_pretrained_checkpoints.sh

Data pkls for Fine-tuning (Response Selection)

Original version for each dataset is availble in Ubuntu Corpus V1, Douban Corpus, and E-Commerce Corpus, respectively.

sh scripts/download_datasets.sh

Domain-specific Post-Training

Post-training Creation

Data for post-training BERT

#Ubuntu Corpus V1
sh scripts/create_bert_post_data_creation_ubuntu.sh
#Douban Corpus
sh scripts/create_bert_post_data_creation_douban.sh
#E-commerce Corpus
sh scripts/create_bert_post_data_creation_e-commerce.sh

Data for post-training ELECTRA

sh scripts/download_electra_post_training_pkl.sh

Post-training Examples

BERT+ (e.g., Ubuntu Corpus V1)

python3 main.py --model bert_post_training --task_name ubuntu --data_dir data/ubuntu_corpus_v1 --bert_pretrained bert-base-uncased --bert_checkpoint_path bert-base-uncased-pytorch_model.bin --task_type response_selection --gpu_ids "0" --root_dir /path/to/root_dir --training_type post_training

ELECTRA+ (e.g., Douban Corpus)

python3 main.py --model electra_post_training --task_name douban --data_dir data/electra_post_training --bert_pretrained electra-base-chinese --bert_checkpoint_path electra-base-chinese-pytorch_model.bin --task_type response_selection --gpu_ids "0" --root_dir /path/to/root_dir --training_type post_training

Training Response Selection Models

Model Arguments

BERT-Base

task_name	data_dir	bert_pretrained	bert_checkpoint_path
ubuntu	data/ubuntu_corpus_v1	bert-base-uncased	bert-base-uncased-pytorch_model.bin
douban e-commerce	data/douban data/e-commerce	bert-base-wwm-chinese	bert-base-wwm-chinese_model.bin

BERT-Post

task_name	data_dir	bert_pretrained	bert_checkpoint_path
ubuntu	data/ubuntu_corpus_v1	bert-post-uncased	bert-post-uncased-pytorch_model.pth
douban	data/douban	bert-post-douban	bert-post-douban-pytorch_model.pth
e-commerce	data/e-commerce	bert-post-ecommerce	bert-post-ecommerce-pytorch_model.pth

ELECTRA-Base

task_name	data_dir	bert_pretrained	bert_checkpoint_path
ubuntu	data/ubuntu_corpus_v1	electra-base	electra-base-pytorch_model.bin
douban e-commerce	data/douban data/e-commerce	electra-base-chinese	electra-base-chinese-pytorch_model.bin

ELECTRA-Post

task_name	data_dir	bert_pretrained	bert_checkpoint_path
ubuntu	data/ubuntu_corpus_v1	electra-post	electra-post-pytorch_model.pth
douban	data/douban	electra-post-douban	electra-post-douban-pytorch_model.pth
e-commerce	data/e-commerce	electra-post-ecommerce	electra-post-ecommerce-pytorch_model.pth

Fine-tuning Examples

BERT+ (e.g., Ubuntu Corpus V1)

python3 main.py --model bert_post --task_name ubuntu --data_dir data/ubuntu_corpus_v1 --bert_pretrained bert-post-uncased --bert_checkpoint_path bert-post-uncased-pytorch_model.pth --task_type response_selection --gpu_ids "0" --root_dir /path/to/root_dir

UMS BERT+ (e.g., Douban Corpus)

python3 main.py --model bert_post --task_name douban --data_dir data/douban --bert_pretrained bert-post-douban --bert_checkpoint_path bert-post-douban-pytorch_model.pth --task_type response_selection --gpu_ids "0" --root_dir /path/to/root_dir --multi_task_type "ins,del,srch"

UMS ELECTRA (e.g., E-Commerce)

python3 main.py --model electra_base --task_name e-commerce --data_dir data/e-commerce --bert_pretrained electra-base-chinese --bert_checkpoint_path electra-base-chinese-pytorch_model.bin --task_type response_selection --gpu_ids "0" --root_dir /path/to/root_dir --multi_task_type "ins,del,srch"

Evaluation

To evaluate the model, set --evaluate to /path/to/checkpoints

UMS BERT+ (e.g., Ubuntu Corpus V1)

python3 main.py --model bert_post --task_name ubuntu --data_dir data/ubuntu_corpus_v1 --bert_pretrained bert-post-uncased --bert_checkpoint_path bert-post-uncased-pytorch_model.pth --task_type response_selection --gpu_ids "0" --root_dir /path/to/root_dir --evaluate /path/to/checkpoints --multi_task_type "ins,del,srch"

Performance

We provide model checkpoints of UMS-BERT+, which obtained new state-of-the-art, for each dataset.

Ubuntu	[email protected]	[email protected]	[email protected]
UMS-BERT+	0.875	0.942	0.988

Douban	MAP	MRR	[email protected]	[email protected]	[email protected]	[email protected]
UMS-BERT+	0.625	0.664	0.499	0.318	0.482	0.858

E-Commerce	[email protected]	[email protected]	[email protected]
UMS-BERT+	0.762	0.905	0.986

PyTorch Implementation for AAAI'21 "Do Response Selection Models Really Know What's Next? Utterance Manipulation Strategies for Multi-turn Response Selection"

Related tags

Overview

UMS for Multi-turn Response Selection

Setup and Dependencies

Anaconda or Miniconda

Preparing Data and Checkpoints

Pre- and Post-trained Checkpoints

Data pkls for Fine-tuning (Response Selection)

Domain-specific Post-Training

Post-training Creation

Data for post-training BERT

Data for post-training ELECTRA

Post-training Examples

BERT+ (e.g., Ubuntu Corpus V1)

ELECTRA+ (e.g., Douban Corpus)

Training Response Selection Models

Model Arguments

BERT-Base

BERT-Post

ELECTRA-Base

ELECTRA-Post

Fine-tuning Examples

BERT+ (e.g., Ubuntu Corpus V1)

UMS BERT+ (e.g., Douban Corpus)

UMS ELECTRA (e.g., E-Commerce)

Evaluation

UMS BERT+ (e.g., Ubuntu Corpus V1)

Performance

Owner

Taesun Whang

Training deep models using anime, illustration images.

Final term project for Bayesian Machine Learning Lecture (XAI-623)

ShinRL: A Library for Evaluating RL Algorithms from Theoretical and Practical Perspectives

[WWW 2022] Zero-Shot Stance Detection via Contrastive Learning

A supplementary code for Editable Neural Networks, an ICLR 2020 submission.

An executor that performs image segmentation on fashion items

Implementation of ConvMixer for "Patches Are All You Need? 🤷"

A CNN model to detect hand gestures.

Official Tensorflow implementation of U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation (ICLR 2020)

ICCV2021: Code for 'Spatial Uncertainty-Aware Semi-Supervised Crowd Counting'

Object detection and instance segmentation toolkit based on PaddlePaddle.

PyTorch implementation for the ICLR 2020 paper "Understanding the Limitations of Variational Mutual Information Estimators"

RMTD: Robust Moving Target Defence Against False Data Injection Attacks in Power Grids

End-to-end face detection, cropping, norm estimation, and landmark detection in a single onnx model

The implementation of the algorithm in the paper "Safe Deep Semi-Supervised Learning for Unseen-Class Unlabeled Data" published in ICML 2020.

A TensorFlow implementation of SOFA, the Simulator for OFfline LeArning and evaluation.

Deep universal probabilistic programming with Python and PyTorch

A novel Engagement Detection with Multi-Task Training (ED-MTT) system

This repo contains the implementation of the algorithm proposed in Off-Belief Learning, ICML 2021.

ChebLieNet, a spectral graph neural network turned equivariant by Riemannian geometry on Lie groups.