fastNLP: A Modularized and Extensible NLP Framework. Currently still in incubation.

Overview

fastNLP

Build Status codecov Pypi Hex.pm Documentation Status

fastNLP是一款轻量级的自然语言处理(NLP)工具包,目标是快速实现NLP任务以及构建复杂模型。

fastNLP具有如下的特性:

  • 统一的Tabular式数据容器,简化数据预处理过程;
  • 内置多种数据集的Loader和Pipe,省去预处理代码;
  • 各种方便的NLP工具,例如Embedding加载(包括ELMo和BERT)、中间数据cache等;
  • 部分数据集与预训练模型的自动下载;
  • 提供多种神经网络组件以及复现模型(涵盖中文分词、命名实体识别、句法分析、文本分类、文本匹配、指代消解、摘要等任务);
  • Trainer提供多种内置Callback函数,方便实验记录、异常捕获等。

安装指南

fastNLP 依赖以下包:

  • numpy>=1.14.2
  • torch>=1.0.0
  • tqdm>=4.28.1
  • nltk>=3.4.1
  • requests
  • spacy
  • prettytable>=0.7.2

其中torch的安装可能与操作系统及 CUDA 的版本相关,请参见 PyTorch 官网 。 在依赖包安装完成后,您可以在命令行执行如下指令完成安装

pip install fastNLP
python -m spacy download en

fastNLP教程

中文文档教程

快速入门

详细使用教程

扩展教程

内置组件

大部分用于的 NLP 任务神经网络都可以看做由词嵌入(embeddings)和两种模块:编码器(encoder)、解码器(decoder)组成。

以文本分类任务为例,下图展示了一个BiLSTM+Attention实现文本分类器的模型流程图:

fastNLP 在 embeddings 模块中内置了几种不同的embedding:静态embedding(GloVe、word2vec)、上下文相关embedding (ELMo、BERT)、字符embedding(基于CNN或者LSTM的CharEmbedding)

与此同时,fastNLP 在 modules 模块中内置了两种模块的诸多组件,可以帮助用户快速搭建自己所需的网络。 两种模块的功能和常见组件如下:

类型 功能 例子
encoder 将输入编码为具有具有表示能力的向量 Embedding, RNN, CNN, Transformer, ...
decoder 将具有某种表示意义的向量解码为需要的输出形式 MLP, CRF, ...

项目结构

fastNLP的大致工作流程如上图所示,而项目结构如下:

fastNLP 开源的自然语言处理库
fastNLP.core 实现了核心功能,包括数据处理组件、训练器、测试器等
fastNLP.models 实现了一些完整的神经网络模型
fastNLP.modules 实现了用于搭建神经网络模型的诸多组件
fastNLP.embeddings 实现了将序列index转为向量序列的功能,包括读取预训练embedding等
fastNLP.io 实现了读写功能,包括数据读入与预处理,模型读写,数据与模型自动下载等

In memory of @FengZiYjun. May his soul rest in peace. We will miss you very very much!

Comments
  • star-transformer何时可以放出完整代码?实验完全无法重现,SST-5数据集上相差6个点哦

    star-transformer何时可以放出完整代码?实验完全无法重现,SST-5数据集上相差6个点哦

    Describe the bug A clear and concise description of what the bug is. 清晰而简要地描述bug

    To Reproduce 使用你们的star-transformer代码,然后用allennlp做训练(glove 42B 词向量), 最后结果见如图,与论文中报告的结果相差6个点。

    请求解释!以及完整版的代码,就是可以完全复现结果的完整版。

    Additional context Add any other context about the problem here. 备注 image

    opened by michael-wzhu 10
  • RuntimeError: CUDA error: device-side assert triggered

    RuntimeError: CUDA error: device-side assert triggered

    Describe the bug 用Predictor方法去加载训练好的模型,在预测时会出现第一张图里面的错误,这个bug被我fixed了。详细请见我在下文上传的项目链接。 出现原因:经过debug分析,发现此bug是由于预测新数据时出现了训练时候没有的新字符,而在bert_embedding.py 脚本里面读取的是训练时候的Vocab维度,并把它初始化成1的vocab向量做mask预测,而这导致了此向量的维度小于实际维度,实际维度=训练时候的Vocab维度+新字符的维度。 Bug结果请看图一,Bug位置及修复请看图二。 image

    image

    To Reproduce 1.把test.txt、dev.txt、train.txt移到data目录下。data目录为自己创建的目录 2. 调用fastNLP_trainer.py脚本 3. 调用fastNLP_predictor.py脚本 4. See error 重现这个bug的步骤

    项目链接:https://github.com/Chris-cbc/fastNLP_Bug_Report_And_Fix.git

    Expected behavior image 上图也是bug修复后出现的结果

    Desktop

    • OS: windows10
    • Python Version: 3.6

    Additional context 请项目主确认后 发邮件并at我github账户一下,让我知道这个bug最终是怎样被修复的 备注

    opened by Chris-cbc 9
  • fastNLP安装完成之后导入有错

    fastNLP安装完成之后导入有错

    Python 3.5环境下安装fastNLP,显示可以安装成功,但是import fastNLP时会出现 File "D:\anaconda\lib\site-packages\fastNLP\core\instance.py", line 40 f" type={(str(type(self.fields[field_name]))).split(s)[1]}" for field_name in self.fields) + "}" ^ SyntaxError: invalid syntax Python3.6和Python3.7也不行,都是安装完成之后,import时就会报错

    opened by lovelyvivi 8
  • a new function for argparse

    a new function for argparse

    we should provide a function for arg parse so that we can support "python fastnlp.py --arg1 value1 --arg2 value2" and so on.

    in this way, what argument should we have?

    enhancement 
    opened by xuyige 8
  • 在运行matching_esim.py时报错RuntimeError: CUDA error: device-side assert triggered

    在运行matching_esim.py时报错RuntimeError: CUDA error: device-side assert triggered

    使用cpu训练没有问题,刚开始以为是pytorch版本问题,后来尝试了1.2、1.4、1.7,其中1.2和1.4都会报错,都是训练到第二个epoch在test时报错RuntimeError: CUDA error: device-side assert triggered。1.7会提示由于pytorch版本问题,对超出词表的词要使用long型。 同样的问题在之前的一个脚本中也出现了。我在4月份使用bertMatching模型训练跑通了,但是现在再做的时候也会报这个错误。 感谢项目组。

    opened by jwc19890114 7
  • Default value for train args.

    Default value for train args.

    https://github.com/fastnlp/fastNLP/blob/8a87807274735046a48be8eb4b1ca10801875039/fastNLP/core/trainer.py#L42-L45

    Should we set some default value for train_args? Otherwise we will pass all these args every time, which is very redundant.

    opened by keezen 7
  • 关于Trainer基本使用部分实例的报错

    关于Trainer基本使用部分实例的报错

    在学习Trainer部分的时候,运行了这一节最开始部分的代码 但是原始的实例代码会报错

    TypeError: can't convert np.ndarray of type numpy.int32. The only supported types are: float64, float32, float16, int64, int32, int16, int8, uint8, and bool.
    

    我尝试在数据生成部分直接使用torch生成tensor

    def generate_psedo_dataset(num_samples):
        data=torch.randint(2,size=(num_samples,10))
        print(data.shape)
        list=[]
        for n in range(num_samples):
            label=torch.sum(data[n])%2
            list.append(label)
        list=torch.stack(list)
        dataset = DataSet({'x':data, 'label': list})
        dataset.set_input('x')
        dataset.set_target('label')
        return dataset
    tr_dataset=generate_psedo_dataset(1000)
    dev_dataset=generate_psedo_dataset(100)
    

    但是在训练中会报如下错误

    TypeError: issubclass() arg 1 must be a class
    

    是不是我的数据生成写错了。。。 gitbook部分的实例代码应该如何调整呢? torch:1.2.0+cu92 FastNLP:0.5.0

    opened by jwc19890114 6
  • 以BertEmbedding为基础进行上层应用训练,训练中更新bert参数的问题。

    以BertEmbedding为基础进行上层应用训练,训练中更新bert参数的问题。

    首先感谢你们的代码!

    我现在想直接利用from fastNLP.embeddings import BertEmbedding来读入BertEmbedding模型,然后根据你们的教程搭建一个vocab,初始化self.embed = BertEmbedding(vocab, model_dir_or_name="en-base-uncased"),进而输入一个句子的中每个词在vocab中对应的index得到对应的embedding向量,然后在此基础上进行后续的语言应用的建模。

    简单来讲,使用方式是否如同pytorch提供的nn.Embedding一样,有什么需要注意的吗?因为我利用上述方式简单搭建了一个baseline,但是并不能很好的收敛。

    还望不吝赐教,谢谢!

    opened by Reply1999 6
  • A question in crf.py

    A question in crf.py

    您好,我发现在decoder的crf.py的代码中,第263行是这样写的 score = trans_score + emit_score[:seq_len - 1, :] 其中的trans_score大小为[seq_len-1, batch_size],trans_score[0][0]代表第0个句子的第0个字符到第1个字符的转移得分; 而emit_score[:seq_len - 1, :]的大小为[seq_len-1, batch_size],emit_score[0, 0]代表第0个句子第0个字符的发射得分; 但是第0个句子第0个字符的转移得分不应该是start字符到第0个字符的score么?请问这里为什么不写成 score = trans_score + emit_score[1:, :]呢 感谢您的解答~

    opened by tyistyler 5
  • [fix]修复fitlocallback 在DistTrainer的使用中无法添加dev_data的问题

    [fix]修复fitlocallback 在DistTrainer的使用中无法添加dev_data的问题

    Description:fitlocallback 在DistTrainer的使用中无法添加dev_data,主要原因在于fitlogcallback验证self.trainer.dev_data时,也即DIstanbulTrainer没有dev_data属性导致调用失败报错

    Main reason: 修复fitlocallback 在DistTrainer的使用中无法添加dev_data的问题

    Checklist 检查下面各项是否完成

    Please feel free to remove inapplicable items for your PR.

    • [x] The PR title starts with [$CATEGORY] (例如[bugfix]修复bug,[new]添加新功能,[test]修改测试,[rm]删除旧代码)
    • [x] Changes are complete (i.e. I finished coding on this PR) 修改完成才提PR
    • [x] All changes have test coverage 修改的部分顺利通过测试。对于fastnlp/fastnlp/的修改,测试代码必须提供在fastnlp/test/
    • [x] Code is well-documented 注释写好,API文档会从注释中抽取
    • [x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change 修改导致例子或tutorial有变化,请找核心开发人员

    Changes: 修复fitlocallback 在DistTrainer的使用中无法添加dev_data的问题

    • 并在DIstTrainer中添加了kwargs,test_use_tqdm,dev_data,metrics类变量

    Mention: 找人review你的PR

    @修改过这个文件的人 @核心开发人员

    opened by ROGERDJQ 5
  • improve the compatibility of

    improve the compatibility of "Trainer"

    Description:简要描述这次PR的内容 Delete DEFAULT_CHECK_BATCH_SIZE and make it same with the input batch size.

    Main reason: 做出这次修改的原因 It is unnecessary to use DEFAULT_CHECK_BATCH_SIZE, which may cause some conficts with the initialized model.

    Checklist 检查下面各项是否完成

    Please feel free to remove inapplicable items for your PR.

    • [x] The PR title starts with [$CATEGORY] (例如[bugfix]修复bug,[new]添加新功能,[test]修改测试,[rm]删除旧代码)
    • [x] Changes are complete (i.e. I finished coding on this PR) 修改完成才提PR
    • [x] All changes have test coverage 修改的部分顺利通过测试。对于fastnlp/fastnlp/的修改,测试代码必须提供在fastnlp/test/
    • [x] Code is well-documented 注释写好,API文档会从注释中抽取
    • [x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change 修改导致例子或tutorial有变化,请找核心开发人员

    Changes: 逐项描述修改的内容

    • 去掉 DEFAULT_CHECK_BATCH_SIZE,并将其修改为预先设置的batch_size

    Mention: 找人review你的PR

    @修改过这个文件的人 @核心开发人员

    opened by hendrydong 5
  • [bugfix] 修改requentments.txt中的rich版本,给topk_saver增加参数

    [bugfix] 修改requentments.txt中的rich版本,给topk_saver增加参数

    Description:修改requentments.txt中的rich版本,给topk_saver增加参数。

    Main reason: 升级rich版本以解决剩余时间过长导致的异常。topk_saver中增加参数use_timestamp_path参数,决定是否跳过创建时间戳命名的文件夹的步骤。

    Checklist 检查下面各项是否完成

    Please feel free to remove inapplicable items for your PR.

    • [x] The PR title starts with [$CATEGORY] (例如[bugfix]修复bug,[new]添加新功能,[test]修改测试,[rm]删除旧代码)
    • [x] Changes are complete (i.e. I finished coding on this PR) 修改完成才提PR
    • [x] All changes have test coverage 修改的部分顺利通过测试。对于fastnlp/fastnlp/的修改,测试代码必须提供在fastnlp/test/
    • [x] Code is well-documented 注释写好,API文档会从注释中抽取
    • [x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change 修改导致例子或tutorial有变化,请找核心开发人员

    Changes: 逐项描述修改的内容

    • 将rich版本由11.2.0升级为12.6.0;
    • topk_saver中增加参数use_timestamp_path参数,决定是否跳过创建时间戳命名的文件夹的步骤。
    opened by 00INDEX 0
  • 版本 1.0.1 No module named 'fastNLP.embeddings.embedding'

    版本 1.0.1 No module named 'fastNLP.embeddings.embedding'

    在 fastNLP 版本 1.0.1 中

    from fastNLP.embeddings.embedding import TokenEmbedding
    

    报错:

    ModuleNotFoundError: No module named 'fastNLP.embeddings.embedding'
    
    opened by MrRace 1
  • 文档疑似错误?

    文档疑似错误?

    https://github.com/fastnlp/fastNLP/blob/6f21084dafeeb937e137adcf33a0858dec921f8c/fastNLP/core/drivers/torch_driver/initialize_torch_driver.py#L36-L37 older ?

    opened by iamqiz 0
  • [疑问][建议]话说为什么DataSet不支持List[Dict]的data?建议像huggingface 的Dataset那样支持一下?

    [疑问][建议]话说为什么DataSet不支持List[Dict]的data?建议像huggingface 的Dataset那样支持一下?

    fastNLP不支持

    from fastNLP import DataSet
    ds=[{"name":"aa","age":21},{"name":"bb","age":22},{"name":"cc","age":19}]
    data_set = DataSet(ds)
    

    huggingface 的Dataset支持

    from datasets import Dataset
    ds=[{"name":"aa","age":21},{"name":"bb","age":22},{"name":"cc","age":19}]
    dataset = Dataset.from_list(ds)
    

    希望能支持

    opened by iamqiz 1
Releases(v0.6.0)
Owner
fastNLP
由复旦大学的自然语言处理(NLP)团队发起的国产自然语言处理开源项目
fastNLP
LOT: A Benchmark for Evaluating Chinese Long Text Understanding and Generation

LOT: A Benchmark for Evaluating Chinese Long Text Understanding and Generation Tasks | Datasets | LongLM | Baselines | Paper Introduction LOT is a ben

46 Dec 28, 2022
Задания КЕГЭ по информатике 2021 на Python

КЕГЭ 2021 на Python В этом репозитории мои решения типовых заданий КЕГЭ по информатике в 2021 году, БЕСПЛАТНО! Задания Взяты с https://inf-ege.sdamgia

8 Oct 13, 2022
Conversational-AI-ChatBot - Intelligent ChatBot built with Microsoft's DialoGPT transformer to make conversations with human users!

Conversational AI ChatBot Intelligent ChatBot built with Microsoft's DialoGPT transformer to make conversations with human users! In this project? Thi

Rajkumar Lakshmanamoorthy 6 Nov 30, 2022
Collection of useful (to me) python scripts for interacting with napari

Napari scripts A collection of napari related tools in various state of disrepair/functionality. Browse_LIF_widget.py This module can be imported, for

5 Aug 15, 2022
Yodatranslator is a simple translator English to Yoda-language

yodatranslator Overview yodatranslator is a simple translator English to Yoda-language. Project is created for educational purposes. It is intended to

1 Nov 11, 2021
Code release for "COTR: Correspondence Transformer for Matching Across Images"

COTR: Correspondence Transformer for Matching Across Images This repository contains the inference code for COTR. We plan to release the training code

UBC Computer Vision Group 358 Dec 24, 2022
A Japanese tokenizer based on recurrent neural networks

Nagisa is a python module for Japanese word segmentation/POS-tagging. It is designed to be a simple and easy-to-use tool. This tool has the following

325 Jan 05, 2023
Interpretable Models for NLP using PyTorch

This repo is deprecated. Please find the updated package here. https://github.com/EdGENetworks/anuvada Anuvada: Interpretable Models for NLP using PyT

Sandeep Tammu 19 Dec 17, 2022
In this project, we aim to achieve the task of predicting emojis from tweets. We aim to investigate the relationship between words and emojis.

Making Emojis More Predictable by Karan Abrol, Karanjot Singh and Pritish Wadhwa, Natural Language Processing (CSE546) under the guidance of Dr. Shad

Karanjot Singh 2 Jan 17, 2022
This project is part of Eleuther AI's quest to create a massive repository of high quality text data for training language models.

This project is part of Eleuther AI's quest to create a massive repository of high quality text data for training language models.

EleutherAI 42 Dec 13, 2022
Twitter Sentiment Analysis using #tag, words and username

Twitter Sentment Analysis Web App using #tag, words and username to fetch data finds Insides of data and Tells Sentiment of the perticular #tag, words or username.

Kumar Saksham 26 Dec 25, 2022
PyTorch Implementation of the paper Single Image Texture Translation for Data Augmentation

SITT The repo contains official PyTorch Implementation of the paper Single Image Texture Translation for Data Augmentation. Authors: Boyi Li Yin Cui T

Boyi Li 52 Jan 05, 2023
Multi Task Vision and Language

12-in-1: Multi-Task Vision and Language Representation Learning Please cite the following if you use this code. Code and pre-trained models for 12-in-

Meta Research 711 Jan 08, 2023
Finding Label and Model Errors in Perception Data With Learned Observation Assertions

Finding Label and Model Errors in Perception Data With Learned Observation Assertions This is the project page for Finding Label and Model Errors in P

Stanford Future Data Systems 17 Oct 14, 2022
Label data using HuggingFace's transformers and automatically get a prediction service

Label Studio for Hugging Face's Transformers Website • Docs • Twitter • Join Slack Community Transfer learning for NLP models by annotating your textu

Heartex 135 Dec 29, 2022
This project converts your human voice input to its text transcript and to an automated voice too.

Human Voice to Automated Voice & Text Introduction: In this project, whenever you'll speak, it will turn your voice into a robot voice and furthermore

Hassan Shahzad 3 Oct 15, 2021
Code Implementation of "Learning Span-Level Interactions for Aspect Sentiment Triplet Extraction".

Span-ASTE: Learning Span-Level Interactions for Aspect Sentiment Triplet Extraction ***** New March 31th, 2022: Scikit-Style API for Easy Usage *****

Chia Yew Ken 111 Dec 23, 2022
Question and answer retrieval in Turkish with BERT

trfaq Google supported this work by providing Google Cloud credit. Thank you Google for supporting the open source! 🎉 What is this? At this repo, I'm

M. Yusuf Sarıgöz 13 Oct 10, 2022
Model for recasing and repunctuating ASR transcripts

Recasing and punctuation model based on Bert Benoit Favre 2021 This system converts a sequence of lowercase tokens without punctuation to a sequence o

Benoit Favre 88 Dec 29, 2022
Prithivida 690 Jan 04, 2023