PyTorch Implementation for AAAI'21 "Do Response Selection Models Really Know What's Next? Utterance Manipulation Strategies for Multi-turn Response Selection"

Last update: Nov 22, 2022

Related tags

Deep Learning UMS-ResSel

Overview

UMS for Multi-turn Response Selection

Implements the model described in the following paper Do Response Selection Models Really Know What's Next? Utterance Manipulation Strategies for Multi-turn Response Selection.

@inproceedings{whang2021ums,
  title={Do Response Selection Models Really Know What's Next? Utterance Manipulation Strategies for Multi-turn Response Selection},
  author={Whang, Taesun and Lee, Dongyub and Oh, Dongsuk and Lee, Chanhee and Han, Kijong and Lee, Dong-hun and Lee, Saebyeok},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  year={2021}
}

This code is reimplemented as a fork of huggingface/transformers and taesunwhang/BERT-ResSel.

Setup and Dependencies

This code is implemented using PyTorch v1.6.0, and provides out of the box support with CUDA 10.1 and CuDNN 7.6.5.

Anaconda / Miniconda is the recommended to set up this codebase.

Anaconda or Miniconda

Clone this repository and create an environment:

git clone https://www.github.com/taesunwhang/UMS-ResSel
conda create -n ums_ressel python=3.7

# activate the environment and install all dependencies
conda activate ums_ressel
cd UMS-ResSel

# https://pytorch.org
pip install torch==1.6.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html
pip install -r requirements.txt

Preparing Data and Checkpoints

Pre- and Post-trained Checkpoints

We provide following pre- and post-trained checkpoints.

bert-base (english), bert-base-wwm (chinese)
bert-post (ubuntu, douban, e-commerce)
electra-base (english), electra-base (chinese)
electra-post (ubuntu, douban, e-commerce)

sh scripts/download_pretrained_checkpoints.sh

Data pkls for Fine-tuning (Response Selection)

Original version for each dataset is availble in Ubuntu Corpus V1, Douban Corpus, and E-Commerce Corpus, respectively.

sh scripts/download_datasets.sh

Domain-specific Post-Training

Post-training Creation

Data for post-training BERT

#Ubuntu Corpus V1
sh scripts/create_bert_post_data_creation_ubuntu.sh
#Douban Corpus
sh scripts/create_bert_post_data_creation_douban.sh
#E-commerce Corpus
sh scripts/create_bert_post_data_creation_e-commerce.sh

Data for post-training ELECTRA

sh scripts/download_electra_post_training_pkl.sh

Post-training Examples

BERT+ (e.g., Ubuntu Corpus V1)

python3 main.py --model bert_post_training --task_name ubuntu --data_dir data/ubuntu_corpus_v1 --bert_pretrained bert-base-uncased --bert_checkpoint_path bert-base-uncased-pytorch_model.bin --task_type response_selection --gpu_ids "0" --root_dir /path/to/root_dir --training_type post_training

ELECTRA+ (e.g., Douban Corpus)

python3 main.py --model electra_post_training --task_name douban --data_dir data/electra_post_training --bert_pretrained electra-base-chinese --bert_checkpoint_path electra-base-chinese-pytorch_model.bin --task_type response_selection --gpu_ids "0" --root_dir /path/to/root_dir --training_type post_training

Training Response Selection Models

Model Arguments

BERT-Base

task_name	data_dir	bert_pretrained	bert_checkpoint_path
ubuntu	data/ubuntu_corpus_v1	bert-base-uncased	bert-base-uncased-pytorch_model.bin
douban e-commerce	data/douban data/e-commerce	bert-base-wwm-chinese	bert-base-wwm-chinese_model.bin

BERT-Post

task_name	data_dir	bert_pretrained	bert_checkpoint_path
ubuntu	data/ubuntu_corpus_v1	bert-post-uncased	bert-post-uncased-pytorch_model.pth
douban	data/douban	bert-post-douban	bert-post-douban-pytorch_model.pth
e-commerce	data/e-commerce	bert-post-ecommerce	bert-post-ecommerce-pytorch_model.pth

ELECTRA-Base

task_name	data_dir	bert_pretrained	bert_checkpoint_path
ubuntu	data/ubuntu_corpus_v1	electra-base	electra-base-pytorch_model.bin
douban e-commerce	data/douban data/e-commerce	electra-base-chinese	electra-base-chinese-pytorch_model.bin

ELECTRA-Post

task_name	data_dir	bert_pretrained	bert_checkpoint_path
ubuntu	data/ubuntu_corpus_v1	electra-post	electra-post-pytorch_model.pth
douban	data/douban	electra-post-douban	electra-post-douban-pytorch_model.pth
e-commerce	data/e-commerce	electra-post-ecommerce	electra-post-ecommerce-pytorch_model.pth

Fine-tuning Examples

BERT+ (e.g., Ubuntu Corpus V1)

python3 main.py --model bert_post --task_name ubuntu --data_dir data/ubuntu_corpus_v1 --bert_pretrained bert-post-uncased --bert_checkpoint_path bert-post-uncased-pytorch_model.pth --task_type response_selection --gpu_ids "0" --root_dir /path/to/root_dir

UMS BERT+ (e.g., Douban Corpus)

python3 main.py --model bert_post --task_name douban --data_dir data/douban --bert_pretrained bert-post-douban --bert_checkpoint_path bert-post-douban-pytorch_model.pth --task_type response_selection --gpu_ids "0" --root_dir /path/to/root_dir --multi_task_type "ins,del,srch"

UMS ELECTRA (e.g., E-Commerce)

python3 main.py --model electra_base --task_name e-commerce --data_dir data/e-commerce --bert_pretrained electra-base-chinese --bert_checkpoint_path electra-base-chinese-pytorch_model.bin --task_type response_selection --gpu_ids "0" --root_dir /path/to/root_dir --multi_task_type "ins,del,srch"

Evaluation

To evaluate the model, set --evaluate to /path/to/checkpoints

UMS BERT+ (e.g., Ubuntu Corpus V1)

python3 main.py --model bert_post --task_name ubuntu --data_dir data/ubuntu_corpus_v1 --bert_pretrained bert-post-uncased --bert_checkpoint_path bert-post-uncased-pytorch_model.pth --task_type response_selection --gpu_ids "0" --root_dir /path/to/root_dir --evaluate /path/to/checkpoints --multi_task_type "ins,del,srch"

Performance

We provide model checkpoints of UMS-BERT+, which obtained new state-of-the-art, for each dataset.

Ubuntu	[email protected]	[email protected]	[email protected]
UMS-BERT+	0.875	0.942	0.988

Douban	MAP	MRR	[email protected]	[email protected]	[email protected]	[email protected]
UMS-BERT+	0.625	0.664	0.499	0.318	0.482	0.858

E-Commerce	[email protected]	[email protected]	[email protected]
UMS-BERT+	0.762	0.905	0.986

PyTorch Implementation for AAAI'21 "Do Response Selection Models Really Know What's Next? Utterance Manipulation Strategies for Multi-turn Response Selection"

Related tags

Overview

UMS for Multi-turn Response Selection

Setup and Dependencies

Anaconda or Miniconda

Preparing Data and Checkpoints

Pre- and Post-trained Checkpoints

Data pkls for Fine-tuning (Response Selection)

Domain-specific Post-Training

Post-training Creation

Data for post-training BERT

Data for post-training ELECTRA

Post-training Examples

BERT+ (e.g., Ubuntu Corpus V1)

ELECTRA+ (e.g., Douban Corpus)

Training Response Selection Models

Model Arguments

BERT-Base

BERT-Post

ELECTRA-Base

ELECTRA-Post

Fine-tuning Examples

BERT+ (e.g., Ubuntu Corpus V1)

UMS BERT+ (e.g., Douban Corpus)

UMS ELECTRA (e.g., E-Commerce)

Evaluation

UMS BERT+ (e.g., Ubuntu Corpus V1)

Performance

Owner

Taesun Whang

Welcome to The Eigensolver Quantum School, a quantum computing crash course designed by students for students.

Project of 'TBEFN: A Two-branch Exposure-fusion Network for Low-light Image Enhancement '

Evolution Strategies in PyTorch

This repository contains codes of ICCV2021 paper: SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation

Code for our TKDE paper "Understanding WeChat User Preferences and “Wow” Diffusion"

Very Deep Convolutional Networks for Large-Scale Image Recognition

Instance Segmentation by Jointly Optimizing Spatial Embeddings and Clustering Bandwidth

Dimension Reduced Turbulent Flow Data From Deep Vector Quantizers

Yolov5 deepsort inference，使用YOLOv5+Deepsort实现车辆行人追踪和计数，代码封装成一个Detector类，更容易嵌入到自己的项目中

Reproduce partial features of DeePMD-kit using PyTorch.

Fast Learning of MNL Model From General Partial Rankings with Application to Network Formation Modeling

implementation of the paper "MarginGAN: Adversarial Training in Semi-Supervised Learning"

Constrained Language Models Yield Few-Shot Semantic Parsers

Official repository for the NeurIPS 2021 paper Get Fooled for the Right Reason: Improving Adversarial Robustness through a Teacher-guided curriculum Learning Approach

An educational resource to help anyone learn deep reinforcement learning.

Spectral Tensor Train Parameterization of Deep Learning Layers

Perturbed Self-Distillation: Weakly Supervised Large-Scale Point Cloud Semantic Segmentation (ICCV2021)

Implementation of ProteinBERT in Pytorch

Transformer part of 12th place solution in Riiid! Answer Correctness Prediction

True Few-Shot Learning with Language Models