Almost State-of-the-art Text Generation library

Last update: Jun 24, 2022

Overview

Ps: we are adding transformer model soon

Text Gen 🐐

Almost State-of-the-art Text Generation library

Text gen is a python library that allow you build a custom text generation model with ease 😄 Something sweet built with Tensorflow and Pytorch(coming soon) - This is the brain of Rosalove ai (https://rosalove.xyz/)

How to use it

Install text-gen

pip install -U text-gen

import the library

from text_gen import ten_textgen as ttg

Load your data. your data must be in a text format.

Download the example data from the example folder

load data

data = 'rl.csv'
text = ttg.loaddata(data)

build our Model Architeture

pipeline = ttg.tentext(text)
seq_text = pipeline.sequence(padding_method = 'pre')
configg = pipeline.configmodel(seq_text, lstmlayer = 128, activation = 'softmax', dropout = 0.25)

train model

model_history = pipeline.fit(loss = 'categorical_crossentropy', optimizer = 'adam', batch = 300, metrics = 'accuracy', epochs = 500, verbose = 0, patience = 10)

generate text using the phrase

pipeline.predict('hello love', word_length = 200, segment = True)

plot loss and accuracy

pipeline.plot_loss_accuracy()

Hyper parameter optimization

Tune your model to know the best optimizer, activation method to use.

pipeline.hyper_params(epochs = 500)

pipeline.saveModel('model')

use a saved model for prediction

#the corpus is the train text file
ttg.load_model_predict(corpus = corpus, padding_method = 'pre', modelname = '../input/model2/model2textgen.h5', sample_text = 'yo yo', word_length = 100)

Give us a star ⭐ 🐉

If you want to contribute, take a look at the issues and the Futurework.md file

Contributors

Comments

use pipenv for managing dependencies

Consider using (pipenv)[https://pypi.org/project/pipenv/] to pin your dependencies. This would allow contributors to easily reproduce the project without messing up the dependencies and its also good on the long run for maintainability

opened by paularah 1
[Snyk] Security upgrade pillow from 6.2.2 to 8.3.2
Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

Changes included in this PR

Changes to the following files to upgrade the vulnerable dependencies to a fixed version:

requirements.txt

Vulnerabilities that will be fixed

By pinning:

Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- | 661/1000
Why? Recently disclosed, Has a fix available, CVSS 7.5 | Regular Expression Denial of Service (ReDoS)
SNYK-PYTHON-PILLOW-1319443 | pillow:
6.2.2 -> 8.3.2
| No | No Known Exploit

(*) Note that the real score may have changed since the PR was raised.

Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.

Check the changes in this PR to ensure they won't cause issues with your project.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

For more information: 🧐 View latest project report

🛠 Adjust project settings

📚 Read more about Snyk's upgrade and patch logic
opened by snyk-bot 0
Read on how to create a simple python library

https://towardsdatascience.com/how-to-build-your-first-python-package-6a00b02635c9

https://medium.com/analytics-vidhya/how-to-create-a-python-library-7d5aea80cc3f

opened by Emekaborisama 0
[Snyk] Security upgrade wheel from 0.30.0 to 0.38.0
This PR was automatically created by Snyk using the credentials of a real user.

Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

Changes included in this PR

Changes to the following files to upgrade the vulnerable dependencies to a fixed version:

requirements.txt

⚠️ Warning

torchvision 0.5.0 requires pillow, which is not installed. tensorflow 1.14.0 requires protobuf, which is not installed. tensorflow-serving-api 1.12.0 requires protobuf, which is not installed. tensorboard 1.14.0 requires protobuf, which is not installed. GPyOpt 1.2.6 requires GPy, which is not installed.

Vulnerabilities that will be fixed

By pinning:

Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- | 551/1000
Why? Recently disclosed, Has a fix available, CVSS 5.3 | Regular Expression Denial of Service (ReDoS)
SNYK-PYTHON-WHEEL-3180413 | wheel:
0.30.0 -> 0.38.0
| No | No Known Exploit

(*) Note that the real score may have changed since the PR was raised.

Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

Check the changes in this PR to ensure they won't cause issues with your project.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

For more information: 🧐 View latest project report

🛠 Adjust project settings

📚 Read more about Snyk's upgrade and patch logic

Learn how to fix vulnerabilities with free interactive lessons:

🦉 Regular Expression Denial of Service (ReDoS)
opened by Emekaborisama 0
[Snyk] Security upgrade wheel from 0.30.0 to 0.38.0
This PR was automatically created by Snyk using the credentials of a real user.

Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

Changes included in this PR

Changes to the following files to upgrade the vulnerable dependencies to a fixed version:

requirements.txt

⚠️ Warning

torchvision 0.5.0 requires pillow, which is not installed. tensorflow 1.14.0 requires grpcio, which is not installed. tensorflow 1.14.0 requires protobuf, which is not installed. tensorboard 1.14.0 requires protobuf, which is not installed. tensorboard 1.14.0 requires grpcio, which is not installed. parameter-sherpa 1.0.6 requires pymongo, which is not installed. parameter-sherpa 1.0.6 requires GPyOpt, which is not installed.

Vulnerabilities that will be fixed

By pinning:

Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- | 551/1000
Why? Recently disclosed, Has a fix available, CVSS 5.3 | Regular Expression Denial of Service (ReDoS)
SNYK-PYTHON-WHEEL-3092128 | wheel:
0.30.0 -> 0.38.0
| No | No Known Exploit

(*) Note that the real score may have changed since the PR was raised.

Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

Check the changes in this PR to ensure they won't cause issues with your project.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

For more information: 🧐 View latest project report

🛠 Adjust project settings

📚 Read more about Snyk's upgrade and patch logic

Learn how to fix vulnerabilities with free interactive lessons:

🦉 Regular Expression Denial of Service (ReDoS)
opened by Emekaborisama 0
[Snyk] Fix for 23 vulnerabilities
Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

Changes included in this PR

Changes to the following files to upgrade the vulnerable dependencies to a fixed version:

requirements.txt

⚠️ Warning

torchvision 0.5.0 requires pillow, which is not installed. parameter-sherpa 1.0.6 requires scikit-learn, which is not installed. GPy 1.10.0 requires paramz, which is not installed. GPy 1.10.0 requires cython, which is not installed. GPy 1.10.0 has requirement scipy<1.5.0,>=1.3.0, but you have scipy 1.2.3.

Vulnerabilities that will be fixed

By pinning:

Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- | 589/1000
Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
SNYK-PYTHON-PILLOW-1055461 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 589/1000
Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
SNYK-PYTHON-PILLOW-1055462 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 509/1000
Why? Has a fix available, CVSS 5.9 | Out-of-bounds Write
SNYK-PYTHON-PILLOW-1059090 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 589/1000
Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
SNYK-PYTHON-PILLOW-1080635 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 589/1000
Why? Has a fix available, CVSS 7.5 | Regular Expression Denial of Service (ReDoS)
SNYK-PYTHON-PILLOW-1080654 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 589/1000
Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
SNYK-PYTHON-PILLOW-1081494 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 589/1000
Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
SNYK-PYTHON-PILLOW-1081501 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 589/1000
Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
SNYK-PYTHON-PILLOW-1081502 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 654/1000
Why? Has a fix available, CVSS 8.8 | Heap-based Buffer Overflow
SNYK-PYTHON-PILLOW-1082329 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 589/1000
Why? Has a fix available, CVSS 7.5 | Insufficient Validation
SNYK-PYTHON-PILLOW-1082750 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 589/1000
Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
SNYK-PYTHON-PILLOW-1090584 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 589/1000
Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
SNYK-PYTHON-PILLOW-1090586 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 589/1000
Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
SNYK-PYTHON-PILLOW-1090587 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 589/1000
Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
SNYK-PYTHON-PILLOW-1090588 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 589/1000
Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
SNYK-PYTHON-PILLOW-1292150 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 589/1000
Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
SNYK-PYTHON-PILLOW-1292151 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 566/1000
Why? Recently disclosed, Has a fix available, CVSS 5.6 | Buffer Overflow
SNYK-PYTHON-PILLOW-1316216 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 414/1000
Why? Has a fix available, CVSS 4 | Out-of-Bounds
SNYK-PYTHON-PILLOW-574573 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 414/1000
Why? Has a fix available, CVSS 4 | Out-of-bounds Read
SNYK-PYTHON-PILLOW-574574 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 414/1000
Why? Has a fix available, CVSS 4 | Out-of-bounds Read
SNYK-PYTHON-PILLOW-574575 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 414/1000
Why? Has a fix available, CVSS 4 | Out-of-bounds Read
SNYK-PYTHON-PILLOW-574576 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 469/1000
Why? Has a fix available, CVSS 5.1 | Buffer Overflow
SNYK-PYTHON-PILLOW-574577 | pillow:
6.2.2 -> 8.1.0
| No | No Known Exploit | 506/1000
Why? Proof of Concept exploit, Has a fix available, CVSS 3.7 | Regular Expression Denial of Service (ReDoS)
SNYK-PYTHON-SCIKITLEARN-1079100 | scikit-learn:
0.20.4 -> 0.24.2
| No | Proof of Concept

(*) Note that the real score may have changed since the PR was raised.

Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.

Check the changes in this PR to ensure they won't cause issues with your project.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

For more information: 🧐 View latest project report

🛠 Adjust project settings

📚 Read more about Snyk's upgrade and patch logic
opened by snyk-bot 0

Releases(v1.9.0)

v1.9.0(Mar 25, 2021)

Source code(tar.gz)
Source code(zip)

Owner

Emeka boris ama

Machine Learning Engineer, Data Scientist, Youtuber and Advocacy

GitHub Repository

Ptorch NLU, a Chinese text classification and sequence annotation toolkit, supports multi class and multi label classification tasks of Chinese long text and short text, and supports sequence annotation tasks such as Chinese named entity recognition, part of speech tagging and word segmentation.

Pytorch-NLU，一个中文文本分类、序列标注工具包，支持中文长文本、短文本的多类、多标签分类任务，支持中文命名实体识别、词性标注、分词等序列标注任务。 Ptorch NLU, a Chinese text classification and sequence annotation toolkit, supports multi class and multi label classifi

186 Dec 24, 2022

Almost State-of-the-art Text Generation library

Related tags

Overview

Text Gen 🐐

Almost State-of-the-art Text Generation library

How to use it

load data

build our Model Architeture

train model

generate text using the phrase

plot loss and accuracy

Hyper parameter optimization

use a saved model for prediction

Give us a star ⭐ 🐉

Comments

use pipenv for managing dependencies

[Snyk] Security upgrade pillow from 6.2.2 to 8.3.2

Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

Changes included in this PR

Vulnerabilities that will be fixed

By pinning:

Read on how to create a simple python library

[Snyk] Security upgrade wheel from 0.30.0 to 0.38.0

Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

Changes included in this PR

Vulnerabilities that will be fixed

By pinning:

[Snyk] Security upgrade wheel from 0.30.0 to 0.38.0

Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

Changes included in this PR

Vulnerabilities that will be fixed

By pinning:

[Snyk] Fix for 23 vulnerabilities

Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

Changes included in this PR

Vulnerabilities that will be fixed

By pinning:

Releases(v1.9.0)

v1.9.0(Mar 25, 2021)

Owner

Emeka boris ama

Ptorch NLU, a Chinese text classification and sequence annotation toolkit, supports multi class and multi label classification tasks of Chinese long text and short text, and supports sequence annotation tasks such as Chinese named entity recognition, part of speech tagging and word segmentation.

Yet Another Neural Machine Translation Toolkit

Baseline code for Korean open domain question answering(ODQA)

基于Transformer的单模型、多尺度的VAE模型

Datasets of Automatic Keyphrase Extraction

Open-Source Toolkit for End-to-End Speech Recognition leveraging PyTorch-Lightning and Hydra.

Espial is an engine for automated organization and discovery of personal knowledge

code for modular summarization work published in ACL2021 by Krishna et al

Conditional probing: measuring usable information beyond a baseline

A benchmark for evaluation and comparison of various NLP tasks in Persian language.

A PyTorch implementation of paper "Learning Shared Semantic Space for Speech-to-Text Translation", ACL (Findings) 2021

kochat

Indobenchmark are collections of Natural Language Understanding (IndoNLU) and Natural Language Generation (IndoNLG)

Chinese Pre-Trained Language Models (CPM-LM) Version-I

Utilities for preprocessing text for deep learning with Keras

Code for Emergent Translation in Multi-Agent Communication

A crowdsourced dataset of dialogues grounded in social contexts involving utilization of commonsense.

Text to speech for Vietnamese, ez to use, ez to update

Japanese NLP Library

Kestrel Threat Hunting Language