A model library for exploring state-of-the-art deep learning topologies and techniques for optimizing Natural Language Processing neural networks

Overview



A Deep Learning NLP/NLU library by Intel® AI Lab

GitHub Website GitHub release

Overview | Models | Installation | Examples | Documentation | Tutorials | Contributing

NLP Architect is an open source Python library for exploring state-of-the-art deep learning topologies and techniques for optimizing Natural Language Processing and Natural Language Understanding Neural Networks.

Overview

NLP Architect is an NLP library designed to be flexible, easy to extend, allow for easy and rapid integration of NLP models in applications and to showcase optimized models.

Features:

  • Core NLP models used in many NLP tasks and useful in many NLP applications

  • Novel NLU models showcasing novel topologies and techniques

  • Optimized NLP/NLU models showcasing different optimization algorithms on neural NLP/NLU models

  • Model-oriented design:

    • Train and run models from command-line.
    • API for using models for inference in python.
    • Procedures to define custom processes for training, inference or anything related to processing.
    • CLI sub-system for running procedures
  • Based on optimized Deep Learning frameworks:

  • Essential utilities for working with NLP models - Text/String pre-processing, IO, data-manipulation, metrics, embeddings.

Installing NLP Architect

We recommend to install NLP Architect in a new python environment, to use python 3.6+ with up-to-date pip, setuptools and h5py.

Install using pip

Install core library only

pip install nlp-architect

Install from source (Github)

Includes core library, examples, solutions and tutorials:

git clone https://github.com/IntelLabs/nlp-architect.git
cd nlp-architect
pip install -e .  # install in developer mode

Running Examples and Solutions

To run provided examples and solutions please install the library with [all] flag which will install extra packages required. (requires installation from source)

pip install .[all]

Models

NLP models that provide best (or near) in class performance:

Natural Language Understanding (NLU) models that address semantic understanding:

Optimizing NLP/NLU models and misc. optimization techniques:

Solutions (End-to-end applications) using one or more models:

Documentation

Full library documentation of NLP models, algorithms, solutions and instructions on how to run each model can be found on our website.

NLP Architect library design philosophy

NLP Architect is a model-oriented library designed to showcase novel and different neural network optimizations. The library contains NLP/NLU related models per task, different neural network topologies (which are used in models), procedures for simplifying workflows in the library, pre-defined data processors and dataset loaders and misc utilities. The library is designed to be a tool for model development: data pre-process, build model, train, validate, infer, save or load a model.

The main design guidelines are:

  • Deep Learning framework agnostic
  • NLP/NLU models per task
  • Different topologies used in models
  • Showcase End-to-End applications (Solutions) utilizing one or more NLP Architect model
  • Generic dataset loaders, textual data processing utilities, and miscellaneous utilities that support NLP model development (loaders, text processors, io, metrics, etc.)
  • Procedures for defining processes for training, inference, optimization or any kind of elaborate script.
  • Pythonic API for using models for inference
  • Extensive model documentation and tutorials

Note

NLP Architect is an active space of research and development; Throughout future releases new models, solutions, topologies and framework additions and changes will be made. We aim to make sure all models run with Python 3.6+. We encourage researchers and developers to contribute their work into the library.

Citing

If you use NLP Architect in your research, please use the following citation:

@misc{izsak_peter_2018_1477518,
  title        = {NLP Architect by Intel AI Lab},
  month        = nov,
  year         = 2018,
  doi          = {10.5281/zenodo.1477518},
  url          = {https://doi.org/10.5281/zenodo.1477518}
}

Disclaimer

The NLP Architect is released as reference code for research purposes. It is not an official Intel product, and the level of quality and support may not be as expected from an official product. NLP Architect is intended to be used locally and has not been designed, developed or evaluated for production usage or web-deployment. Additional algorithms and environments are planned to be added to the framework. Feedback and contributions from the open source and NLP research communities are more than welcome.

Contact

Contact the NLP Architect development team through Github issues or email: [email protected]

Issues
  • Runtime errors when running with the ABSA model

    Runtime errors when running with the ABSA model

    I am trying to run the ABSA solution on my local Ubuntu 18.04 even with the provided train reviews dataset but getting the following error (using the exact instructions + virtualenv): 'file_name': ['tripadvisor_co_uk-travel_restaurant_reviews_sample_2000_train.csv']}}], 'references': []}: OSError('Unable to load model. Filepath is not an hdf5 file (or h5py is not available) or SavedModel.',) When I am trying to load a bunch of reviews, I get the following error:

    'references': []}: ParserError('Expected 2 fields in line 2, saw 3')

    Please advise how to fix it.

    bug 
    opened by stevesolun 10
  • Pip install failing from master

    Pip install failing from master

    Target objective:

    pip Install .

    Steps to objective:

    Successfully built nlp-architect spacy 2.0.18 has requirement numpy>=1.15.0, but you'll have numpy 1.14.5 which is incompatible. Installing collected packages: spacy, nlp-architect Killed

    Pull-Request related:

    opened by aribornstein 8
  • bug: Error exporting QuantizedBert to ONNX

    bug: Error exporting QuantizedBert to ONNX

    When attempting to export a Q8BERT model to ONNX, TorchScript reports a runtime error.

    Since the quantized forward methods embed Tensors with grads from layer instance attributes rather than input parameters, exporting the model has to be done with explicit torch script tracing; however that creates a traced model with lacks fidelity with the python model, as branching behavior is not captured.

    However, attempting to create a script model with TorchScript, which should encode that behavior results in a missing config attribute on the BertModel superclass.

    To Reproduce Steps to reproduce the behavior:

    1. Train a Q8Bert model:
     nlp-train transformer_glue \
                --task_name mrpc \
                --model_name_or_path bert-base-uncased \
                --model_type quant_bert \
                --learning_rate 2e-5 \
                --output_dir $DATA_DIR \
                --data_dir $GLUE_DIR  \
                --evaluate_during_training \
                --do_lower_case \
                --per_gpu_train_batch_size 32 \
                --per_gpu_eval_batch_size 32 \
                --max_seq_length 128
    
    1. Load the 8bit model for inference
    import torch
    from nlp_architect.models.transformers.quantized_bert import QuantizedBertForSequenceClassification
    
    model = QuantizedBertForSequenceClassification.from_pretrained(configs.data_dir, from_8bit=True)
    device = torch.device("cuda")
    model.to(device)
    model.eval()
    
    
    1. Attempt to create the script model
    script_model = torch.jit.script(model)
    
    1. This produces the error:
    Traceback (most recent call last):
      File "convert-torch-q8bert-via-onnx-to-tensorflow.py", line 105, in <module>
        script_model = torch.jit.script(model)
      File "/home/krhys/.local/share/virtualenvs/q8bert-_S0V57VU/lib/python3.7/site-packages/torch/jit/__init__.py", line 1203, in script
        return torch.jit.torch.jit._recursive.recursive_script(obj)
      File "/home/krhys/.local/share/virtualenvs/q8bert-_S0V57VU/lib/python3.7/site-packages/torch/jit/_recursive.py", line 173, in recursive_script
        return copy_to_script_module(mod, overload_stubs + stubs)
      File "/home/krhys/.local/share/virtualenvs/q8bert-_S0V57VU/lib/python3.7/site-packages/torch/jit/_recursive.py", line 95, in copy_to_script_module
        torch.jit._create_methods_from_stubs(script_module, stubs)
      File "/home/krhys/.local/share/virtualenvs/q8bert-_S0V57VU/lib/python3.7/site-packages/torch/jit/__init__.py", line 1423, in _create_methods_from_stubs
        self._c._create_methods(self, defs, rcbs, defaults)
      File "/home/krhys/.local/share/virtualenvs/q8bert-_S0V57VU/lib/python3.7/site-packages/torch/jit/_recursive.py", line 195, in make_strong_submodule
        new_strong_submodule = recursive_script(module)
      File "/home/krhys/.local/share/virtualenvs/q8bert-_S0V57VU/lib/python3.7/site-packages/torch/jit/_recursive.py", line 173, in recursive_script
        return copy_to_script_module(mod, overload_stubs + stubs)
      File "/home/krhys/.local/share/virtualenvs/q8bert-_S0V57VU/lib/python3.7/site-packages/torch/jit/_recursive.py", line 95, in copy_to_script_module
        torch.jit._create_methods_from_stubs(script_module, stubs)
      File "/home/krhys/.local/share/virtualenvs/q8bert-_S0V57VU/lib/python3.7/site-packages/torch/jit/__init__.py", line 1423, in _create_methods_from_stubs
        self._c._create_methods(self, defs, rcbs, defaults)
    RuntimeError:
    module has no attribute 'config':
    at /home/krhys/.local/share/virtualenvs/q8bert-_S0V57VU/lib/python3.7/site-packages/transformers/modeling_bert.py:675:15
            # We can provide a self-attention mask of dimensions [batch_size, from_seq_length, to_seq_length]
            # ourselves in which case we just need to make it broadcastable to all heads.
            if attention_mask.dim() == 3:
                extended_attention_mask = attention_mask[:, None, :, :]
    
            # Provided a padding mask of dimensions [batch_size, seq_length]
            # - if the model is a decoder, apply a causal mask in addition to the padding mask
            # - if the model is an encoder, make the mask broadcastable to [batch_size, num_heads, seq_length, seq_length]
            if attention_mask.dim() == 2:
                if self.config.is_decoder:
                   ~~~~~~~~~~~ <--- HERE
                    batch_size, seq_length = input_shape
                    seq_ids = torch.arange(seq_length, device=device)
                    causal_mask = seq_ids[None, None, :].repeat(batch_size, seq_length, 1) <= seq_ids[None, :, None]
                    extended_attention_mask = causal_mask[:, None, :, :] * attention_mask[:, None, None, :]
                else:
                    extended_attention_mask = attention_mask[:, None, None, :]
    
            # Since attention_mask is 1.0 for positions we want to attend and 0.0 for
            # masked positions, this operation will create a tensor which is 0.0 for
    '__torch__.nlp_architect.models.transformers.quantized_bert.QuantizedBertModel.forward' is being compiled since it was called from '__torch__.nlp_architect.models.transformers.quantized_bert.QuantizedBertForSequenceClassification.forward'
    at /home/krhys/.local/share/virtualenvs/q8bert-_S0V57VU/lib/python3.7/site-packages/transformers/modeling_bert.py:1014:8
        def forward(self, input_ids=None, attention_mask=None, token_type_ids=None,
                    position_ids=None, head_mask=None, inputs_embeds=None, labels=None):
    
            outputs = self.bert(input_ids,
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~...  <--- HERE
                                attention_mask=attention_mask,
                                token_type_ids=token_type_ids,
                                position_ids=position_ids,
                                head_mask=head_mask,
                                inputs_embeds=inputs_embeds)
    
            pooled_output = outputs[1]
    
            pooled_output = self.dropout(pooled_output)
    

    Expected behavior

    It should produce a script model without error

    **Environment setup: **

    • OS (Linux/Mac OS): Ubuntu Xenial
    • Python version: 3.7

    Additional context

    It may be possible to create pure helper quantized forward functions with explicit arguments which have the @torch.jit.script annotations which are called by the quantized layer forward methods

    Note that the HuggingFace transformers Bert model can be exported to ONNX.

    bug Stale 
    opened by kirstin-rhys 7
  • Any readme for BERT example?

    Any readme for BERT example?

    Target objective:

    Want help to run https://github.com/NervanaSystems/nlp-architect/blob/master/nlp_architect/procedures/transformers/glue.py

    Hi, I want to run the bert example newly added to the repo. Wondering is there any readme for running the example or any documentation for bert/quantized-bert related features? For instance, where to download model and 'labels.txt'?

    Thanks so much for your help in advance!

    opened by hkvision 7
  • nlp_architect command fails if dev req aren't installed

    nlp_architect command fails if dev req aren't installed

    Target objective:

    nlp_architect command fails if dev req aren't installed detect is libraries required are install and print a msg that states the error + how to install

    Steps to objective:

    • [ ] fix cmd.py
    • [ ]

    Pull-Request related:

    bug 
    opened by peteriz 7
  • GPU training support

    GPU training support

    This is more of a question than issue but I wasn't able to find a forum to ask the question.

    I see installation instruction on enabling GPU supports. Yet, many algorithms (e.g. ner) do not specifically have a parameter for GPU enabling. Is it an issue easy to get around with or is it going to be a feature supported in the future?

    question 
    opened by DihuiLai 7
  • Problem running bist FileNotFoundError: [Errno 2] No such file or directory:  (...) params.json

    Problem running bist FileNotFoundError: [Errno 2] No such file or directory: (...) params.json

    I tried to run bist in your interface using the command :

    python3 server/serve.py --name bist

    I get the following error :

    25MB [00:28,  1.16s/MB]                                                                                                                                                                                     
    Download Complete
    Unzipping...
    Done.
    Traceback (most recent call last):
      File "serve.py", line 306, in <module>
        set_server_properties(app, args.name)
      File "serve.py", line 287, in set_server_properties
        service = Service(service_name)
      File "serve.py", line 143, in __init__
        self.service = self.load_service(service_name)
      File "serve.py", line 259, in load_service
        upload_service.load_model()
      File "/home/catalina/nlp-architect/nlp_architect/api/bist_parser_api.py", line 32, in load_model
        self.model = SpacyBISTParser()
      File "/home/catalina/nlp-architect/nlp_architect/pipelines/spacy_bist.py", line 48, in __init__
        self.bist_parser.load(bist_model if bist_model else SpacyBISTParser.pretrained)
      File "/home/catalina/nlp-architect/nlp_architect/models/bist_parser.py", line 117, in load
        with open(os.path.join(os.path.dirname(path), 'params.json'), 'r') as file:
    FileNotFoundError: [Errno 2] No such file or directory: '/home/catalina/nlp-architect/nlp_architect/pipelines/bist-pretrained/params.json'
    
    opened by Catadanna 7
  • How to improve bist model accuracy? :Bistmodel

    How to improve bist model accuracy? :Bistmodel

    I'm using spacybistparser with my own spacy and bist model trained using universal dependencies v2.0 dataset. My training follows the doc. I want to improve the accuracy of bistmodel. Currently, with default parameters, I train the bist model for 20iterations. The evaluation uas score is 87.8 percent. How can I improve it to 93.8 as mentioned in the docs? Is it possible with this english ud 2.0 dataset? Please help.

    question 
    opened by ravishpankar 6
  • question about Quantize BERT?

    question about Quantize BERT?

    opened by RyanHuangNLP 6
  • Question about Seq2SeqIntentModel

    Question about Seq2SeqIntentModel

    This is used for slot tagging, and in the example you provide, the dataset is SNIPS and it is used for NER. Wondering why it is placed under intent_extraction and come with the name IntentModel?

    question Stale 
    opened by hkvision 6
  • Getting ValueError('Length  must be less than 1000') error

    Getting ValueError('Length must be less than 1000') error

    I am trying to run ABSA with the a CSV and getting this error. I have sent the file to @danielkorat. It contains 1000 rows and it fails with the above error.

    Please advise how can it be solved. Until now 500K reviews (in one file) have been sent and it worked.

    bug 
    opened by stevesolun 5
  • Bump numpy from 1.18.4 to 1.22.0 in /solutions/InterpreT/application

    Bump numpy from 1.18.4 to 1.22.0 in /solutions/InterpreT/application

    Bumps numpy from 1.18.4 to 1.22.0.

    Release notes

    Sourced from numpy's releases.

    v1.22.0

    NumPy 1.22.0 Release Notes

    NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are:

    • Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release.
    • A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX.
    • NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data.
    • New methods for quantile, percentile, and related functions. The new methods provide a complete set of the methods commonly found in the literature.
    • A new configurable allocator for use by downstream projects.

    These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation.

    The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays.

    Expired deprecations

    Deprecated numeric style dtype strings have been removed

    Using the strings "Bytes0", "Datetime64", "Str0", "Uint32", and "Uint64" as a dtype will now raise a TypeError.

    (gh-19539)

    Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio

    numpy.loads was deprecated in v1.15, with the recommendation that users use pickle.loads instead. ndfromtxt and mafromtxt were both deprecated in v1.17 - users should use numpy.genfromtxt instead with the appropriate value for the usemask parameter.

    (gh-19615)

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies python 
    opened by dependabot[bot] 0
  • Bump numpy from 1.17 to 1.22.0 in /solutions/absa_solution

    Bump numpy from 1.17 to 1.22.0 in /solutions/absa_solution

    Bumps numpy from 1.17 to 1.22.0.

    Release notes

    Sourced from numpy's releases.

    v1.22.0

    NumPy 1.22.0 Release Notes

    NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are:

    • Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release.
    • A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX.
    • NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data.
    • New methods for quantile, percentile, and related functions. The new methods provide a complete set of the methods commonly found in the literature.
    • A new configurable allocator for use by downstream projects.

    These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation.

    The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays.

    Expired deprecations

    Deprecated numeric style dtype strings have been removed

    Using the strings "Bytes0", "Datetime64", "Str0", "Uint32", and "Uint64" as a dtype will now raise a TypeError.

    (gh-19539)

    Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio

    numpy.loads was deprecated in v1.15, with the recommendation that users use pickle.loads instead. ndfromtxt and mafromtxt were both deprecated in v1.17 - users should use numpy.genfromtxt instead with the appropriate value for the usemask parameter.

    (gh-19615)

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies python 
    opened by dependabot[bot] 0
  • Bump numpy from 1.17 to 1.22.0

    Bump numpy from 1.17 to 1.22.0

    Bumps numpy from 1.17 to 1.22.0.

    Release notes

    Sourced from numpy's releases.

    v1.22.0

    NumPy 1.22.0 Release Notes

    NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are:

    • Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release.
    • A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX.
    • NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data.
    • New methods for quantile, percentile, and related functions. The new methods provide a complete set of the methods commonly found in the literature.
    • A new configurable allocator for use by downstream projects.

    These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation.

    The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays.

    Expired deprecations

    Deprecated numeric style dtype strings have been removed

    Using the strings "Bytes0", "Datetime64", "Str0", "Uint32", and "Uint64" as a dtype will now raise a TypeError.

    (gh-19539)

    Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio

    numpy.loads was deprecated in v1.15, with the recommendation that users use pickle.loads instead. ndfromtxt and mafromtxt were both deprecated in v1.17 - users should use numpy.genfromtxt instead with the appropriate value for the usemask parameter.

    (gh-19615)

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies python 
    opened by dependabot[bot] 0
  • Bump tensorflow from 1.15.2 to 2.7.2 in /solutions/absa_solution

    Bump tensorflow from 1.15.2 to 2.7.2 in /solutions/absa_solution

    Bumps tensorflow from 1.15.2 to 2.7.2.

    Release notes

    Sourced from tensorflow's releases.

    TensorFlow 2.7.2

    Release 2.7.2

    This releases introduces several vulnerability fixes:

    TensorFlow 2.7.1

    Release 2.7.1

    This releases introduces several vulnerability fixes:

    • Fixes a floating point division by 0 when executing convolution operators (CVE-2022-21725)
    • Fixes a heap OOB read in shape inference for ReverseSequence (CVE-2022-21728)
    • Fixes a heap OOB access in Dequantize (CVE-2022-21726)
    • Fixes an integer overflow in shape inference for Dequantize (CVE-2022-21727)
    • Fixes a heap OOB access in FractionalAvgPoolGrad (CVE-2022-21730)
    • Fixes an overflow and divide by zero in UnravelIndex (CVE-2022-21729)
    • Fixes a type confusion in shape inference for ConcatV2 (CVE-2022-21731)
    • Fixes an OOM in ThreadPoolHandle (CVE-2022-21732)
    • Fixes an OOM due to integer overflow in StringNGrams (CVE-2022-21733)
    • Fixes more issues caused by incomplete validation in boosted trees code (CVE-2021-41208)
    • Fixes an integer overflows in most sparse component-wise ops (CVE-2022-23567)
    • Fixes an integer overflows in AddManySparseToTensorsMap (CVE-2022-23568)

    ... (truncated)

    Changelog

    Sourced from tensorflow's changelog.

    Release 2.7.2

    This releases introduces several vulnerability fixes:

    Release 2.6.4

    This releases introduces several vulnerability fixes:

    • Fixes a code injection in saved_model_cli (CVE-2022-29216)
    • Fixes a missing validation which causes TensorSummaryV2 to crash (CVE-2022-29193)
    • Fixes a missing validation which crashes QuantizeAndDequantizeV4Grad (CVE-2022-29192)
    • Fixes a missing validation which causes denial of service via DeleteSessionTensor (CVE-2022-29194)
    • Fixes a missing validation which causes denial of service via GetSessionTensor (CVE-2022-29191)
    • Fixes a missing validation which causes denial of service via StagePeek (CVE-2022-29195)
    • Fixes a missing validation which causes denial of service via UnsortedSegmentJoin (CVE-2022-29197)
    • Fixes a missing validation which causes denial of service via LoadAndRemapMatrix (CVE-2022-29199)
    • Fixes a missing validation which causes denial of service via SparseTensorToCSRSparseMatrix (CVE-2022-29198)
    • Fixes a missing validation which causes denial of service via LSTMBlockCell (CVE-2022-29200)
    • Fixes a missing validation which causes denial of service via Conv3DBackpropFilterV2 (CVE-2022-29196)
    • Fixes a CHECK failure in depthwise ops via overflows (CVE-2021-41197)
    • Fixes issues arising from undefined behavior stemming from users supplying invalid resource handles (CVE-2022-29207)
    • Fixes a segfault due to missing support for quantized types (CVE-2022-29205)
    • Fixes a missing validation which results in undefined behavior in SparseTensorDenseAdd (CVE-2022-29206)

    ... (truncated)

    Commits
    • dd7b8a3 Merge pull request #56034 from tensorflow-jenkins/relnotes-2.7.2-15779
    • 1e7d6ea Update RELEASE.md
    • 5085135 Merge pull request #56069 from tensorflow/mm-cp-52488e5072f6fe44411d70c6af09e...
    • adafb45 Merge pull request #56060 from yongtang:curl-7.83.1
    • 01cb1b8 Merge pull request #56038 from tensorflow-jenkins/version-numbers-2.7.2-4733
    • 8c90c2f Update version numbers to 2.7.2
    • 43f3cdc Update RELEASE.md
    • 98b0a48 Insert release notes place-fill
    • dfa5cf3 Merge pull request #56028 from tensorflow/disable-tests-on-r2.7
    • 501a65c Disable timing out tests
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies python 
    opened by dependabot[bot] 0
  • Bump tensorflow from 1.15.4 to 2.7.2

    Bump tensorflow from 1.15.4 to 2.7.2

    Bumps tensorflow from 1.15.4 to 2.7.2.

    Release notes

    Sourced from tensorflow's releases.

    TensorFlow 2.7.2

    Release 2.7.2

    This releases introduces several vulnerability fixes:

    TensorFlow 2.7.1

    Release 2.7.1

    This releases introduces several vulnerability fixes:

    • Fixes a floating point division by 0 when executing convolution operators (CVE-2022-21725)
    • Fixes a heap OOB read in shape inference for ReverseSequence (CVE-2022-21728)
    • Fixes a heap OOB access in Dequantize (CVE-2022-21726)
    • Fixes an integer overflow in shape inference for Dequantize (CVE-2022-21727)
    • Fixes a heap OOB access in FractionalAvgPoolGrad (CVE-2022-21730)
    • Fixes an overflow and divide by zero in UnravelIndex (CVE-2022-21729)
    • Fixes a type confusion in shape inference for ConcatV2 (CVE-2022-21731)
    • Fixes an OOM in ThreadPoolHandle (CVE-2022-21732)
    • Fixes an OOM due to integer overflow in StringNGrams (CVE-2022-21733)
    • Fixes more issues caused by incomplete validation in boosted trees code (CVE-2021-41208)
    • Fixes an integer overflows in most sparse component-wise ops (CVE-2022-23567)
    • Fixes an integer overflows in AddManySparseToTensorsMap (CVE-2022-23568)

    ... (truncated)

    Changelog

    Sourced from tensorflow's changelog.

    Release 2.7.2

    This releases introduces several vulnerability fixes:

    Release 2.6.4

    This releases introduces several vulnerability fixes:

    • Fixes a code injection in saved_model_cli (CVE-2022-29216)
    • Fixes a missing validation which causes TensorSummaryV2 to crash (CVE-2022-29193)
    • Fixes a missing validation which crashes QuantizeAndDequantizeV4Grad (CVE-2022-29192)
    • Fixes a missing validation which causes denial of service via DeleteSessionTensor (CVE-2022-29194)
    • Fixes a missing validation which causes denial of service via GetSessionTensor (CVE-2022-29191)
    • Fixes a missing validation which causes denial of service via StagePeek (CVE-2022-29195)
    • Fixes a missing validation which causes denial of service via UnsortedSegmentJoin (CVE-2022-29197)
    • Fixes a missing validation which causes denial of service via LoadAndRemapMatrix (CVE-2022-29199)
    • Fixes a missing validation which causes denial of service via SparseTensorToCSRSparseMatrix (CVE-2022-29198)
    • Fixes a missing validation which causes denial of service via LSTMBlockCell (CVE-2022-29200)
    • Fixes a missing validation which causes denial of service via Conv3DBackpropFilterV2 (CVE-2022-29196)
    • Fixes a CHECK failure in depthwise ops via overflows (CVE-2021-41197)
    • Fixes issues arising from undefined behavior stemming from users supplying invalid resource handles (CVE-2022-29207)
    • Fixes a segfault due to missing support for quantized types (CVE-2022-29205)
    • Fixes a missing validation which results in undefined behavior in SparseTensorDenseAdd (CVE-2022-29206)

    ... (truncated)

    Commits
    • dd7b8a3 Merge pull request #56034 from tensorflow-jenkins/relnotes-2.7.2-15779
    • 1e7d6ea Update RELEASE.md
    • 5085135 Merge pull request #56069 from tensorflow/mm-cp-52488e5072f6fe44411d70c6af09e...
    • adafb45 Merge pull request #56060 from yongtang:curl-7.83.1
    • 01cb1b8 Merge pull request #56038 from tensorflow-jenkins/version-numbers-2.7.2-4733
    • 8c90c2f Update version numbers to 2.7.2
    • 43f3cdc Update RELEASE.md
    • 98b0a48 Insert release notes place-fill
    • dfa5cf3 Merge pull request #56028 from tensorflow/disable-tests-on-r2.7
    • 501a65c Disable timing out tests
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies python 
    opened by dependabot[bot] 0
  • Performance issue in the definition of _inference, examples/memn2n_dialogue/memn2n_dialogue.py(P1)

    Performance issue in the definition of _inference, examples/memn2n_dialogue/memn2n_dialogue.py(P1)

    Hello, I found a performance issue in the difinition of_inference, examples/memn2n_dialogue/memn2n_dialogue.py, tf.nn.embedding_lookup will be calculated repeately during the program execution, resulting in reduced efficiency. So I think it should be created before the loop.

    The same issue exist in tf.reduce_sum in line 187 and 200.

    Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.

    bug 
    opened by DLPerf 2
Releases(v0.5.5.1)
  • v0.5.5.1(Nov 17, 2020)

  • v0.5.4.1(Sep 16, 2020)

  • v0.5.4(Apr 12, 2020)

    • Improved ID-CNN and LSTM sequence tagger models accuracy
    • Added support for BILOU format for tagging tasks
    • Updated Transformer token tagging and neural taggers to save best and support BILOU format
    • Refactor pseudo/distillation procedures to reproduce results according to our paper
    Source code(tar.gz)
    Source code(zip)
  • v0.5(Aug 28, 2019)

    General features

    • Added PyTorch as a backend
    • New nlp_architect CLI for training/running/process models and scripts
    • New nlp_architect.procedures package for creating procedures/scripts for doing scripted work with NLP architect.
    • S3 URL caching of pre-trained models
    • Many additions/refactors/base classes reflecting new API (WIP)

    Transformer models

    • Integrated pytorch-transformers library into NLP Architect's model package.
    • Created a base transformer class for training transformer-based models
    • Added sequence/token classification models

    Optimization related models

    • Added a base transformer model for training quantized (8bit) BERT models
    • Added distillation process from Transformer models into CNN/LSTM based token classification models

    NLP related additions

    • Added GLUE benchmark tasks
    • Added CNN-LSTM and ID-CNN token classification models (pytorch backend)

    Documentation

    • Updated documentation website to reflect 0.5 release
    • Updated website with logo and matching style
    Source code(tar.gz)
    Source code(zip)
  • v0.4(Apr 2, 2019)

  • v0.3.1(Dec 17, 2018)

    Improvements

    • Installation process refactored and supports pip install nlp-architect (much more simple)
    • Moved server and solutions into main package
    • Many Cross-doc coreference model fixes and additions
    • Updates and fixed lots of documentation files

    Demos

    • Improved NLP Architect demos with a new design
    • Renamed nlp_architect demo to nlp_architect server
    Source code(tar.gz)
    Source code(zip)
  • v0.3(Nov 1, 2018)

    New Solution

    • Topics and Trend Analysis - extract topics and compare two temporal versions a corpus, highlighting hot and cold trends.

    New models

    • Sparse GNMT - A Tensorflow implementation of the GNMT model with sparsity and quantization operations integrated.
    • Semantic Relation Identification - Extract semantic relation types of two words or phrases using external resources.
    • Sieve-based Cross Document Coreference - A seive-based model for finding similar entities or events across different documents from the same domain.

    Improvements

    • Reading comprehension - added inference mode.
    • Sequential models - updated NER, IE, Chunker models to use tf.keras and added CNN-character based feature extractors and improved accuracy of all models.
    • CRF Layer - added native Tensorflow based CRF layer.
    • Word Sense Disambiguation - model updated to use tf.keras.
    • Demo UI - updated demo UI using AngularJS.
    • Installation - improved installation process and added support for CPU/MKL/GPU backends for Tensorflow.
    • NLP Architect cmd - added nlp_architect - a simple command initiator to handle maintenance tasks, see nlp_architect -h for the list of commands.
    • Lots of bug fixes and refactoring.
    Source code(tar.gz)
    Source code(zip)
  • v0.2(Aug 15, 2018)

    Release v0.2

    New Solution

    • Term Set Expansion - the task of expanding a given partial set of terms into a more complete set of terms that belong to the same semantic class. This solution demonstrates the usage of NLP Architect models (Word Chunker and NP2Vec) used in an application solution.

    New models

    • Unsupervised Crosslingual Embeddings model using a GAN to learn a mapping between languages - implemented in Tensorflow
    • Language Model (LM) using Temporal Convolution Network (TCN) - implemented in Tensorflow
    • Supervised Sentiment Classification - implemented in Keras

    Model improvements

    • Reading comprehension - refactored to use Tensorflow
    • End-to-end Memory Network for Goal Oriented Dialogue - refactored to use Tensorflow
    • Word Chunker - refactored to use tf.keras and use state-of-art model
    • NP semantic segmentation - refactored to use tf.keras
    • Updated CONLL2000, Amazon_Review, PTB, Fasttext, Wikitext-103 and Wikipedia-dump dataset loaders.

    New features

    • REST server refactored to use hug, new streamlined the UI and improved documentation. See updated documentation for further details.
    • Noun Phrase annotator plug-in for spaCy pipeline
    • Publications page with relevant material demonstrating the usage of NLP Architect
    • Tutorials page with a collection of Jupyter notebook tutorials using NLP Architect models
    Source code(tar.gz)
    Source code(zip)
  • 0.1(May 29, 2018)

    The current version of NLP Architect includes these features that we found interesting from both research perspectives and practical applications:

    • NLP core models that allow robust extraction of linguistic features for NLP workflow: for example, dependency parser (BIST) and NP chunker
    • NLU modules that provide best in class performance: for example, intent extraction (IE), name entity recognition (NER)
    • Modules that address semantic understanding: for example, colocations, most common word sense, NP embedding representation (e.g. NP2V)
    • Components instrumental for conversational AI: for example, ChatBot applications, including dialog system, sequence chunking and IE
    • End-to-end DL applications using new topologies: for example, Q&A, machine reading comprehension
    Source code(tar.gz)
    Source code(zip)
A model library for exploring state-of-the-art deep learning topologies and techniques for optimizing Natural Language Processing neural networks

A Deep Learning NLP/NLU library by Intel® AI Lab Overview | Models | Installation | Examples | Documentation | Tutorials | Contributing NLP Architect

Intel Labs 2.8k Jun 27, 2022
🤗Transformers: State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0.

State-of-the-art Natural Language Processing for PyTorch and TensorFlow 2.0 ?? Transformers provides thousands of pretrained models to perform tasks o

Hugging Face 66.2k Jul 1, 2022
🤗Transformers: State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0.

State-of-the-art Natural Language Processing for PyTorch and TensorFlow 2.0 ?? Transformers provides thousands of pretrained models to perform tasks o

Hugging Face 40.9k Feb 18, 2021
🤗 Transformers: State-of-the-art Natural Language Processing for Pytorch, TensorFlow, and JAX.

English | 简体中文 | 繁體中文 State-of-the-art Natural Language Processing for Jax, PyTorch and TensorFlow ?? Transformers provides thousands of pretrained mo

Hugging Face 65.9k Jun 26, 2022
Natural language processing summarizer using 3 state of the art Transformer models: BERT, GPT2, and T5

NLP-Summarizer Natural language processing summarizer using 3 state of the art Transformer models: BERT, GPT2, and T5 This project aimed to provide in

Samuel Sharkey 1 Feb 7, 2022
A very simple framework for state-of-the-art Natural Language Processing (NLP)

A very simple framework for state-of-the-art NLP. Developed by Humboldt University of Berlin and friends. IMPORTANT: (30.08.2020) We moved our models

flair 11.7k Jun 21, 2022
State of the Art Natural Language Processing

Spark NLP: State of the Art Natural Language Processing Spark NLP is a Natural Language Processing library built on top of Apache Spark ML. It provide

John Snow Labs 2.8k Jun 24, 2022
A very simple framework for state-of-the-art Natural Language Processing (NLP)

A very simple framework for state-of-the-art NLP. Developed by Humboldt University of Berlin and friends. IMPORTANT: (30.08.2020) We moved our models

flair 10k Feb 18, 2021
State of the Art Natural Language Processing

Spark NLP: State of the Art Natural Language Processing Spark NLP is a Natural Language Processing library built on top of Apache Spark ML. It provide

John Snow Labs 1.9k Feb 18, 2021
A very simple framework for state-of-the-art Natural Language Processing (NLP)

A very simple framework for state-of-the-art NLP. Developed by Humboldt University of Berlin and friends. Flair is: A powerful NLP library. Flair allo

flair 11.8k Jun 29, 2022
State of the art faster Natural Language Processing in Tensorflow 2.0 .

tf-transformers: faster and easier state-of-the-art NLP in TensorFlow 2.0 ****************************************************************************

null 69 Jun 15, 2022
:mag: End-to-End Framework for building natural language search interfaces to data by utilizing Transformers and the State-of-the-Art of NLP. Supporting DPR, Elasticsearch, HuggingFace’s Modelhub and much more!

Haystack is an end-to-end framework that enables you to build powerful and production-ready pipelines for different search use cases. Whether you want

deepset 1.4k Feb 18, 2021
Named-entity recognition using neural networks. Easy-to-use and state-of-the-art results.

NeuroNER NeuroNER is a program that performs named-entity recognition (NER). Website: neuroner.com. This page gives step-by-step instructions to insta

Franck Dernoncourt 1.6k Jun 24, 2022
Named-entity recognition using neural networks. Easy-to-use and state-of-the-art results.

NeuroNER NeuroNER is a program that performs named-entity recognition (NER). Website: neuroner.com. This page gives step-by-step instructions to insta

Franck Dernoncourt 1.5k Feb 11, 2021
Named-entity recognition using neural networks. Easy-to-use and state-of-the-art results.

NeuroNER NeuroNER is a program that performs named-entity recognition (NER). Website: neuroner.com. This page gives step-by-step instructions to insta

Franck Dernoncourt 1.5k Feb 17, 2021
Indobenchmark are collections of Natural Language Understanding (IndoNLU) and Natural Language Generation (IndoNLG)

Indobenchmark Toolkit Indobenchmark are collections of Natural Language Understanding (IndoNLU) and Natural Language Generation (IndoNLG) resources fo

Samuel Cahyawijaya 9 Jun 20, 2022
DELTA is a deep learning based natural language and speech processing platform.

DELTA - A DEep learning Language Technology plAtform What is DELTA? DELTA is a deep learning based end-to-end natural language and speech processing p

DELTA 1.5k Jun 24, 2022
DELTA is a deep learning based natural language and speech processing platform.

DELTA - A DEep learning Language Technology plAtform What is DELTA? DELTA is a deep learning based end-to-end natural language and speech processing p

DELTA 1.4k Feb 17, 2021