fastNLP: A Modularized and Extensible NLP Framework. Currently still in incubation.

Overview

fastNLP

Build Status codecov Pypi Hex.pm Documentation Status

fastNLP是一款轻量级的自然语言处理(NLP)工具包,目标是快速实现NLP任务以及构建复杂模型。

fastNLP具有如下的特性:

  • 统一的Tabular式数据容器,简化数据预处理过程;
  • 内置多种数据集的Loader和Pipe,省去预处理代码;
  • 各种方便的NLP工具,例如Embedding加载(包括ELMo和BERT)、中间数据cache等;
  • 部分数据集与预训练模型的自动下载;
  • 提供多种神经网络组件以及复现模型(涵盖中文分词、命名实体识别、句法分析、文本分类、文本匹配、指代消解、摘要等任务);
  • Trainer提供多种内置Callback函数,方便实验记录、异常捕获等。

安装指南

fastNLP 依赖以下包:

  • numpy>=1.14.2
  • torch>=1.0.0
  • tqdm>=4.28.1
  • nltk>=3.4.1
  • requests
  • spacy
  • prettytable>=0.7.2

其中torch的安装可能与操作系统及 CUDA 的版本相关,请参见 PyTorch 官网 。 在依赖包安装完成后,您可以在命令行执行如下指令完成安装

pip install fastNLP
python -m spacy download en

fastNLP教程

中文文档教程

快速入门

详细使用教程

扩展教程

内置组件

大部分用于的 NLP 任务神经网络都可以看做由词嵌入(embeddings)和两种模块:编码器(encoder)、解码器(decoder)组成。

以文本分类任务为例,下图展示了一个BiLSTM+Attention实现文本分类器的模型流程图:

fastNLP 在 embeddings 模块中内置了几种不同的embedding:静态embedding(GloVe、word2vec)、上下文相关embedding (ELMo、BERT)、字符embedding(基于CNN或者LSTM的CharEmbedding)

与此同时,fastNLP 在 modules 模块中内置了两种模块的诸多组件,可以帮助用户快速搭建自己所需的网络。 两种模块的功能和常见组件如下:

类型 功能 例子
encoder 将输入编码为具有具有表示能力的向量 Embedding, RNN, CNN, Transformer, ...
decoder 将具有某种表示意义的向量解码为需要的输出形式 MLP, CRF, ...

项目结构

fastNLP的大致工作流程如上图所示,而项目结构如下:

fastNLP 开源的自然语言处理库
fastNLP.core 实现了核心功能,包括数据处理组件、训练器、测试器等
fastNLP.models 实现了一些完整的神经网络模型
fastNLP.modules 实现了用于搭建神经网络模型的诸多组件
fastNLP.embeddings 实现了将序列index转为向量序列的功能,包括读取预训练embedding等
fastNLP.io 实现了读写功能,包括数据读入与预处理,模型读写,数据与模型自动下载等

In memory of @FengZiYjun. May his soul rest in peace. We will miss you very very much!

Comments
  • star-transformer何时可以放出完整代码?实验完全无法重现,SST-5数据集上相差6个点哦

    star-transformer何时可以放出完整代码?实验完全无法重现,SST-5数据集上相差6个点哦

    Describe the bug A clear and concise description of what the bug is. 清晰而简要地描述bug

    To Reproduce 使用你们的star-transformer代码,然后用allennlp做训练(glove 42B 词向量), 最后结果见如图,与论文中报告的结果相差6个点。

    请求解释!以及完整版的代码,就是可以完全复现结果的完整版。

    Additional context Add any other context about the problem here. 备注 image

    opened by michael-wzhu 10
  • RuntimeError: CUDA error: device-side assert triggered

    RuntimeError: CUDA error: device-side assert triggered

    Describe the bug 用Predictor方法去加载训练好的模型,在预测时会出现第一张图里面的错误,这个bug被我fixed了。详细请见我在下文上传的项目链接。 出现原因:经过debug分析,发现此bug是由于预测新数据时出现了训练时候没有的新字符,而在bert_embedding.py 脚本里面读取的是训练时候的Vocab维度,并把它初始化成1的vocab向量做mask预测,而这导致了此向量的维度小于实际维度,实际维度=训练时候的Vocab维度+新字符的维度。 Bug结果请看图一,Bug位置及修复请看图二。 image

    image

    To Reproduce 1.把test.txt、dev.txt、train.txt移到data目录下。data目录为自己创建的目录 2. 调用fastNLP_trainer.py脚本 3. 调用fastNLP_predictor.py脚本 4. See error 重现这个bug的步骤

    项目链接:https://github.com/Chris-cbc/fastNLP_Bug_Report_And_Fix.git

    Expected behavior image 上图也是bug修复后出现的结果

    Desktop

    • OS: windows10
    • Python Version: 3.6

    Additional context 请项目主确认后 发邮件并at我github账户一下,让我知道这个bug最终是怎样被修复的 备注

    opened by Chris-cbc 9
  • fastNLP安装完成之后导入有错

    fastNLP安装完成之后导入有错

    Python 3.5环境下安装fastNLP,显示可以安装成功,但是import fastNLP时会出现 File "D:\anaconda\lib\site-packages\fastNLP\core\instance.py", line 40 f" type={(str(type(self.fields[field_name]))).split(s)[1]}" for field_name in self.fields) + "}" ^ SyntaxError: invalid syntax Python3.6和Python3.7也不行,都是安装完成之后,import时就会报错

    opened by lovelyvivi 8
  • a new function for argparse

    a new function for argparse

    we should provide a function for arg parse so that we can support "python fastnlp.py --arg1 value1 --arg2 value2" and so on.

    in this way, what argument should we have?

    enhancement 
    opened by xuyige 8
  • 在运行matching_esim.py时报错RuntimeError: CUDA error: device-side assert triggered

    在运行matching_esim.py时报错RuntimeError: CUDA error: device-side assert triggered

    使用cpu训练没有问题,刚开始以为是pytorch版本问题,后来尝试了1.2、1.4、1.7,其中1.2和1.4都会报错,都是训练到第二个epoch在test时报错RuntimeError: CUDA error: device-side assert triggered。1.7会提示由于pytorch版本问题,对超出词表的词要使用long型。 同样的问题在之前的一个脚本中也出现了。我在4月份使用bertMatching模型训练跑通了,但是现在再做的时候也会报这个错误。 感谢项目组。

    opened by jwc19890114 7
  • Default value for train args.

    Default value for train args.

    https://github.com/fastnlp/fastNLP/blob/8a87807274735046a48be8eb4b1ca10801875039/fastNLP/core/trainer.py#L42-L45

    Should we set some default value for train_args? Otherwise we will pass all these args every time, which is very redundant.

    opened by keezen 7
  • 关于Trainer基本使用部分实例的报错

    关于Trainer基本使用部分实例的报错

    在学习Trainer部分的时候,运行了这一节最开始部分的代码 但是原始的实例代码会报错

    TypeError: can't convert np.ndarray of type numpy.int32. The only supported types are: float64, float32, float16, int64, int32, int16, int8, uint8, and bool.
    

    我尝试在数据生成部分直接使用torch生成tensor

    def generate_psedo_dataset(num_samples):
        data=torch.randint(2,size=(num_samples,10))
        print(data.shape)
        list=[]
        for n in range(num_samples):
            label=torch.sum(data[n])%2
            list.append(label)
        list=torch.stack(list)
        dataset = DataSet({'x':data, 'label': list})
        dataset.set_input('x')
        dataset.set_target('label')
        return dataset
    tr_dataset=generate_psedo_dataset(1000)
    dev_dataset=generate_psedo_dataset(100)
    

    但是在训练中会报如下错误

    TypeError: issubclass() arg 1 must be a class
    

    是不是我的数据生成写错了。。。 gitbook部分的实例代码应该如何调整呢? torch:1.2.0+cu92 FastNLP:0.5.0

    opened by jwc19890114 6
  • 以BertEmbedding为基础进行上层应用训练,训练中更新bert参数的问题。

    以BertEmbedding为基础进行上层应用训练,训练中更新bert参数的问题。

    首先感谢你们的代码!

    我现在想直接利用from fastNLP.embeddings import BertEmbedding来读入BertEmbedding模型,然后根据你们的教程搭建一个vocab,初始化self.embed = BertEmbedding(vocab, model_dir_or_name="en-base-uncased"),进而输入一个句子的中每个词在vocab中对应的index得到对应的embedding向量,然后在此基础上进行后续的语言应用的建模。

    简单来讲,使用方式是否如同pytorch提供的nn.Embedding一样,有什么需要注意的吗?因为我利用上述方式简单搭建了一个baseline,但是并不能很好的收敛。

    还望不吝赐教,谢谢!

    opened by Reply1999 6
  • A question in crf.py

    A question in crf.py

    您好,我发现在decoder的crf.py的代码中,第263行是这样写的 score = trans_score + emit_score[:seq_len - 1, :] 其中的trans_score大小为[seq_len-1, batch_size],trans_score[0][0]代表第0个句子的第0个字符到第1个字符的转移得分; 而emit_score[:seq_len - 1, :]的大小为[seq_len-1, batch_size],emit_score[0, 0]代表第0个句子第0个字符的发射得分; 但是第0个句子第0个字符的转移得分不应该是start字符到第0个字符的score么?请问这里为什么不写成 score = trans_score + emit_score[1:, :]呢 感谢您的解答~

    opened by tyistyler 5
  • [fix]修复fitlocallback 在DistTrainer的使用中无法添加dev_data的问题

    [fix]修复fitlocallback 在DistTrainer的使用中无法添加dev_data的问题

    Description:fitlocallback 在DistTrainer的使用中无法添加dev_data,主要原因在于fitlogcallback验证self.trainer.dev_data时,也即DIstanbulTrainer没有dev_data属性导致调用失败报错

    Main reason: 修复fitlocallback 在DistTrainer的使用中无法添加dev_data的问题

    Checklist 检查下面各项是否完成

    Please feel free to remove inapplicable items for your PR.

    • [x] The PR title starts with [$CATEGORY] (例如[bugfix]修复bug,[new]添加新功能,[test]修改测试,[rm]删除旧代码)
    • [x] Changes are complete (i.e. I finished coding on this PR) 修改完成才提PR
    • [x] All changes have test coverage 修改的部分顺利通过测试。对于fastnlp/fastnlp/的修改,测试代码必须提供在fastnlp/test/
    • [x] Code is well-documented 注释写好,API文档会从注释中抽取
    • [x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change 修改导致例子或tutorial有变化,请找核心开发人员

    Changes: 修复fitlocallback 在DistTrainer的使用中无法添加dev_data的问题

    • 并在DIstTrainer中添加了kwargs,test_use_tqdm,dev_data,metrics类变量

    Mention: 找人review你的PR

    @修改过这个文件的人 @核心开发人员

    opened by ROGERDJQ 5
  • improve the compatibility of

    improve the compatibility of "Trainer"

    Description:简要描述这次PR的内容 Delete DEFAULT_CHECK_BATCH_SIZE and make it same with the input batch size.

    Main reason: 做出这次修改的原因 It is unnecessary to use DEFAULT_CHECK_BATCH_SIZE, which may cause some conficts with the initialized model.

    Checklist 检查下面各项是否完成

    Please feel free to remove inapplicable items for your PR.

    • [x] The PR title starts with [$CATEGORY] (例如[bugfix]修复bug,[new]添加新功能,[test]修改测试,[rm]删除旧代码)
    • [x] Changes are complete (i.e. I finished coding on this PR) 修改完成才提PR
    • [x] All changes have test coverage 修改的部分顺利通过测试。对于fastnlp/fastnlp/的修改,测试代码必须提供在fastnlp/test/
    • [x] Code is well-documented 注释写好,API文档会从注释中抽取
    • [x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change 修改导致例子或tutorial有变化,请找核心开发人员

    Changes: 逐项描述修改的内容

    • 去掉 DEFAULT_CHECK_BATCH_SIZE,并将其修改为预先设置的batch_size

    Mention: 找人review你的PR

    @修改过这个文件的人 @核心开发人员

    opened by hendrydong 5
  • [bugfix] 修改requentments.txt中的rich版本,给topk_saver增加参数

    [bugfix] 修改requentments.txt中的rich版本,给topk_saver增加参数

    Description:修改requentments.txt中的rich版本,给topk_saver增加参数。

    Main reason: 升级rich版本以解决剩余时间过长导致的异常。topk_saver中增加参数use_timestamp_path参数,决定是否跳过创建时间戳命名的文件夹的步骤。

    Checklist 检查下面各项是否完成

    Please feel free to remove inapplicable items for your PR.

    • [x] The PR title starts with [$CATEGORY] (例如[bugfix]修复bug,[new]添加新功能,[test]修改测试,[rm]删除旧代码)
    • [x] Changes are complete (i.e. I finished coding on this PR) 修改完成才提PR
    • [x] All changes have test coverage 修改的部分顺利通过测试。对于fastnlp/fastnlp/的修改,测试代码必须提供在fastnlp/test/
    • [x] Code is well-documented 注释写好,API文档会从注释中抽取
    • [x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change 修改导致例子或tutorial有变化,请找核心开发人员

    Changes: 逐项描述修改的内容

    • 将rich版本由11.2.0升级为12.6.0;
    • topk_saver中增加参数use_timestamp_path参数,决定是否跳过创建时间戳命名的文件夹的步骤。
    opened by 00INDEX 0
  • 版本 1.0.1 No module named 'fastNLP.embeddings.embedding'

    版本 1.0.1 No module named 'fastNLP.embeddings.embedding'

    在 fastNLP 版本 1.0.1 中

    from fastNLP.embeddings.embedding import TokenEmbedding
    

    报错:

    ModuleNotFoundError: No module named 'fastNLP.embeddings.embedding'
    
    opened by MrRace 1
  • 文档疑似错误?

    文档疑似错误?

    https://github.com/fastnlp/fastNLP/blob/6f21084dafeeb937e137adcf33a0858dec921f8c/fastNLP/core/drivers/torch_driver/initialize_torch_driver.py#L36-L37 older ?

    opened by iamqiz 0
  • [疑问][建议]话说为什么DataSet不支持List[Dict]的data?建议像huggingface 的Dataset那样支持一下?

    [疑问][建议]话说为什么DataSet不支持List[Dict]的data?建议像huggingface 的Dataset那样支持一下?

    fastNLP不支持

    from fastNLP import DataSet
    ds=[{"name":"aa","age":21},{"name":"bb","age":22},{"name":"cc","age":19}]
    data_set = DataSet(ds)
    

    huggingface 的Dataset支持

    from datasets import Dataset
    ds=[{"name":"aa","age":21},{"name":"bb","age":22},{"name":"cc","age":19}]
    dataset = Dataset.from_list(ds)
    

    希望能支持

    opened by iamqiz 1
Releases(v0.6.0)
Owner
fastNLP
由复旦大学的自然语言处理(NLP)团队发起的国产自然语言处理开源项目
fastNLP
lightweight, fast and robust columnar dataframe for data analytics with online update

streamdf Streamdf is a lightweight data frame library built on top of the dictionary of numpy array, developed for Kaggle's time-series code competiti

23 May 19, 2022
CCQA A New Web-Scale Question Answering Dataset for Model Pre-Training

CCQA: A New Web-Scale Question Answering Dataset for Model Pre-Training This is the official repository for the code and models of the paper CCQA: A N

Meta Research 29 Nov 30, 2022
Summarization module based on KoBART

KoBART-summarization Install KoBART pip install git+https://github.com/SKT-AI/KoBART#egg=kobart Requirements pytorch==1.7.0 transformers==4.0.0 pytor

seujung hwan, Jung 148 Dec 28, 2022
Kerberoast with ACL abuse capabilities

targetedKerberoast targetedKerberoast is a Python script that can, like many others (e.g. GetUserSPNs.py), print "kerberoast" hashes for user accounts

Shutdown 213 Dec 22, 2022
CorNet Correlation Networks for Extreme Multi-label Text Classification

CorNet Correlation Networks for Extreme Multi-label Text Classification Prerequisites python==3.6.3 pytorch==1.2.0 torchgpipe==0.0.5 click==7.0 ruamel

Guangxu Xun 38 Dec 31, 2022
nlabel is a library for generating, storing and retrieving tagging information and embedding vectors from various nlp libraries through a unified interface.

nlabel is a library for generating, storing and retrieving tagging information and embedding vectors from various nlp libraries through a unified interface.

Bernhard Liebl 2 Jun 10, 2022
This code is the implementation of Text Emotion Recognition (TER) with linguistic features

APSIPA-TER This code is the implementation of Text Emotion Recognition (TER) with linguistic features. The network model is BERT with a pretrained mod

kenro515 1 Feb 08, 2022
The model is designed to train a single and large neural network in order to predict correct translation by reading the given sentence.

Neural Machine Translation communication system The model is basically direct to convert one source language to another targeted language using encode

Nishant Banjade 7 Sep 22, 2022
An open source framework for seq2seq models in PyTorch.

pytorch-seq2seq Documentation This is a framework for sequence-to-sequence (seq2seq) models implemented in PyTorch. The framework has modularized and

International Business Machines 1.4k Jan 02, 2023
Recognition of 38 speech commands in russian. Based on Yandex Cup 2021 ML Challenge: ASR

Speech_38_ru_commands Recognition of 38 speech commands in russian. Based on Yandex Cup 2021 ML Challenge: ASR Программа умеет распознавать 38 ключевы

Andrey 9 May 05, 2022
A PyTorch Implementation of End-to-End Models for Speech-to-Text

speech Speech is an open-source package to build end-to-end models for automatic speech recognition. Sequence-to-sequence models with attention, Conne

Awni Hannun 647 Dec 25, 2022
A model library for exploring state-of-the-art deep learning topologies and techniques for optimizing Natural Language Processing neural networks

A Deep Learning NLP/NLU library by Intel® AI Lab Overview | Models | Installation | Examples | Documentation | Tutorials | Contributing NLP Architect

Intel Labs 2.9k Dec 31, 2022
XLNet: Generalized Autoregressive Pretraining for Language Understanding

Introduction XLNet is a new unsupervised language representation learning method based on a novel generalized permutation language modeling objective.

Zihang Dai 6k Jan 07, 2023
Generate custom detailed survey paper with topic clustered sections and proper citations, from just a single query in just under 30 mins !!

Auto-Research A no-code utility to generate a detailed well-cited survey with topic clustered sections (draft paper format) and other interesting arti

Sidharth Pal 20 Dec 14, 2022
Officile code repository for "A Game-Theoretic Perspective on Risk-Sensitive Reinforcement Learning"

CvarAdversarialRL Official code repository for "A Game-Theoretic Perspective on Risk-Sensitive Reinforcement Learning". Initial setup Create a virtual

Mathieu Godbout 1 Nov 19, 2021
The training code for the 4th place model at MDX 2021 leaderboard A.

The training code for the 4th place model at MDX 2021 leaderboard A.

Chin-Yun Yu 32 Dec 18, 2022
多语言降噪预训练模型MBart的中文生成任务

mbart-chinese 基于mbart-large-cc25 的中文生成任务 Input source input: text + /s + lang_code target input: lang_code + text + /s Usage token_ids_mapping.jso

11 Sep 19, 2022
Sequence-to-sequence framework with a focus on Neural Machine Translation based on Apache MXNet

Sequence-to-sequence framework with a focus on Neural Machine Translation based on Apache MXNet

Amazon Web Services - Labs 1.1k Dec 27, 2022
Implementation of legal QA system based on SentenceKoBART

LegalQA using SentenceKoBART Implementation of legal QA system based on SentenceKoBART How to train SentenceKoBART Based on Neural Search Engine Jina

Heewon Jeon(gogamza) 75 Dec 27, 2022
Bpe algorithm can finetune tokenizer - Bpe algorithm can finetune tokenizer

"# bpe_algorithm_can_finetune_tokenizer" this is an implyment for https://github

张博 1 Feb 02, 2022