Feature_CRF_AE
Feature_CRF_AE provides a implementation of Bridging Pre-trained Language Models and Hand-crafted Features for Unsupervised POS Tagging:
@inproceedings{zhou-etal-2022-Bridging,
title = {Bridging Pre-trained Language Models and Hand-crafted Features for Unsupervised POS Tagging},
author = {Zhou, houquan and Li, yang and Li, Zhenghua and Zhang Min},
booktitle = {Findings of ACL},
year = {2022},
url = {?},
pages = {?--?}
}
Please concact Jacob_Zhou \at outlook.com if you have any questions.
Contents
Installation
Feature_CRF_AE can be installing from source:
$ git clone https://github.com/Jacob-Zhou/FeatureCRFAE && cd FeatureCRFAE
$ bash scripts/setup.sh
The following requirements will be installed in scripts/setup.sh:
python: 3.7allennlp: 1.2.2pytorch: 1.6.0transformers: 3.5.1h5py: 3.1.0matplotlib: 3.3.1nltk: 3.5numpy: 1.19.1overrides: 3.1.0scikit_learn: 1.0.2seaborn: 0.11.0tqdm: 4.49.0
For WSJ data, we use the ELMo representations of elmo_2x4096_512_2048cnn_2xhighway_5.5B from AllenNLP. For UD data, we use the ELMo representations released by HIT-SCIR.
The corresponding data and ELMo models can be download as follows:
# 1) UD data and ELMo models:
$ bash scripts/prepare_data.sh
# 2) UD data, ELMo models as well as WSJ data
# [please replace ~/treebank3/parsed/mrg/wsj/ with your path to LDC99T42]
$ bash scripts/prepare_data.sh ~/treebank3/parsed/mrg/wsj/
Performance
WSJ-All
| Seed | M-1 | 1-1 | VM |
|---|---|---|---|
| 0 | 84.29 | 70.03 | 78.43 |
| 1 | 82.34 | 64.42 | 77.27 |
| 2 | 84.68 | 62.78 | 77.83 |
| 3 | 82.55 | 65.00 | 77.35 |
| 4 | 82.20 | 66.69 | 77.33 |
| Avg. | 83.21 | 65.78 | 77.64 |
| Std. | 1.18 | 2.75 | 0.49 |
WSJ-Test
| Seed | M-1 | 1-1 | VM |
|---|---|---|---|
| 0 | 81.99 | 64.84 | 76.86 |
| 1 | 82.52 | 61.46 | 76.13 |
| 2 | 82.33 | 61.15 | 75.13 |
| 3 | 78.11 | 58.80 | 72.94 |
| 4 | 82.05 | 61.68 | 76.21 |
| Avg. | 81.40 | 61.59 | 75.45 |
| Std. | 1.85 | 2.15 | 1.54 |
Usage
We give some examples on scripts/examples.sh. Before run the code you should activate the virtual environment by:
$ . scripts/set_environment.sh
Training
To train a model from scratch, it is preferred to use the command-line option, which is more flexible and customizable. Here are some training examples:
$ python -u -m tagger.cmds.crf_ae train \
--conf configs/crf_ae.ini \
--encoder elmo \
--plm elmo_models/allennlp/elmo_2x4096_512_2048cnn_2xhighway_5.5B \
--train data/wsj/total.conll \
--evaluate data/wsj/total.conll \
--path save/crf_ae_wsj
$ python -u -m tagger.cmds.crf_ae train \
--conf configs/crf_ae.ini \
--ud-mode \
--ud-feature \
--ignore-capitalized \
--language-specific-strip \
--feat-min-freq 14 \
--language de \
--encoder elmo \
--plm elmo_models/de \
--train data/ud/de/total.conll \
--evaluate data/ud/de/total.conll \
--path save/crf_ae_de
For more instructions on training, please type python -m tagger.cmds.[crf_ae|feature_hmm] train -h.
Alternatively, We provides some equivalent command entry points registered in setup.py: crf-ae and feature-hmm.
$ crf-ae train \
--conf configs/crf_ae.ini \
--encoder elmo \
--plm elmo_models/allennlp/elmo_2x4096_512_2048cnn_2xhighway_5.5B \
--train data/wsj/total.conll \
--evaluate data/wsj/total.conll \
--path save/crf_ae
Evaluation
$ python -u -m tagger.cmds.crf_ae evaluate \
--conf configs/crf_ae.ini \
--encoder elmo \
--plm elmo_models/allennlp/elmo_2x4096_512_2048cnn_2xhighway_5.5B \
--data data/wsj/total.conll \
--path save/crf_ae
Predict
$ python -u -m tagger.cmds.crf_ae predict \
--conf configs/crf_ae.ini \
--encoder elmo \
--plm elmo_models/allennlp/elmo_2x4096_512_2048cnn_2xhighway_5.5B \
--data data/wsj/total.conll \
--path save/crf_ae \
--pred save/crf_ae/pred.conll
192 Dec 23, 2022
166 Dec 11, 2022
138 Dec 27, 2022
13 Oct 10, 2022
49 Dec 26, 2022
4 Feb 18, 2022
160 Dec 23, 2022
2 Feb 17, 2022
1 Jan 06, 2022
79 Dec 26, 2022
1 Dec 06, 2021
80 Jan 03, 2023
72 Dec 19, 2022
32.9k Jan 08, 2023
399 Jan 05, 2023
61 Jan 01, 2023
169 Jan 05, 2023
1.5k Dec 28, 2022
65 Dec 30, 2022
255 Dec 27, 2022