Almost State-of-the-art Text Generation library

Related tags

Text Data & NLPpythontensorflowtext-analysistext-generationpytorchlstm-neural-networkstext-gen
Overview

Ps: we are adding transformer model soon

Text Gen 🐐

Downloads python tensorflow PyPI

Almost State-of-the-art Text Generation library

Text gen is a python library that allow you build a custom text generation model with ease 😄 Something sweet built with Tensorflow and Pytorch(coming soon) - This is the brain of Rosalove ai (https://rosalove.xyz/)

How to use it

Install text-gen

pip install -U text-gen

import the library

from text_gen import ten_textgen as ttg

Load your data. your data must be in a text format.

Download the example data from the example folder

load data

data = 'rl.csv'
text = ttg.loaddata(data)

build our Model Architeture

pipeline = ttg.tentext(text)
seq_text = pipeline.sequence(padding_method = 'pre')
configg = pipeline.configmodel(seq_text, lstmlayer = 128, activation = 'softmax', dropout = 0.25)

train model

model_history = pipeline.fit(loss = 'categorical_crossentropy', optimizer = 'adam', batch = 300, metrics = 'accuracy', epochs = 500, verbose = 0, patience = 10)

generate text using the phrase

pipeline.predict('hello love', word_length = 200, segment = True)

plot loss and accuracy

pipeline.plot_loss_accuracy()

Hyper parameter optimization

Tune your model to know the best optimizer, activation method to use.

pipeline.hyper_params(epochs = 500)
pipeline.saveModel('model')

use a saved model for prediction

#the corpus is the train text file
ttg.load_model_predict(corpus = corpus, padding_method = 'pre', modelname = '../input/model2/model2textgen.h5', sample_text = 'yo yo', word_length = 100)

Give us a star ⭐ 🐉

If you want to contribute, take a look at the issues and the Futurework.md file

Contributors

Comments
  • use pipenv for managing dependencies

    use pipenv for managing dependencies

    Consider using (pipenv)[https://pypi.org/project/pipenv/] to pin your dependencies. This would allow contributors to easily reproduce the project without messing up the dependencies and its also good on the long run for maintainability

    opened by paularah 1
  • [Snyk] Security upgrade pillow from 6.2.2 to 8.3.2

    [Snyk] Security upgrade pillow from 6.2.2 to 8.3.2

    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 661/1000
    Why? Recently disclosed, Has a fix available, CVSS 7.5 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-PILLOW-1319443 | pillow:
    6.2.2 -> 8.3.2
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    opened by snyk-bot 0
  • Read on how to create a simple python library

    Read on how to create a simple python library

    https://towardsdatascience.com/how-to-build-your-first-python-package-6a00b02635c9

    https://medium.com/analytics-vidhya/how-to-create-a-python-library-7d5aea80cc3f

    opened by Emekaborisama 0
  • [Snyk] Security upgrade wheel from 0.30.0 to 0.38.0

    [Snyk] Security upgrade wheel from 0.30.0 to 0.38.0

    This PR was automatically created by Snyk using the credentials of a real user.


    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt
    ⚠️ Warning 
    torchvision 0.5.0 requires pillow, which is not installed.
tensorflow 1.14.0 requires protobuf, which is not installed.
tensorflow-serving-api 1.12.0 requires protobuf, which is not installed.
tensorboard 1.14.0 requires protobuf, which is not installed.
GPyOpt 1.2.6 requires GPy, which is not installed.

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- medium severity | 551/1000
    Why? Recently disclosed, Has a fix available, CVSS 5.3 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-WHEEL-3180413 | wheel:
    0.30.0 -> 0.38.0
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    Learn how to fix vulnerabilities with free interactive lessons:

    🦉 Regular Expression Denial of Service (ReDoS)

    opened by Emekaborisama 0
  • [Snyk] Security upgrade wheel from 0.30.0 to 0.38.0

    [Snyk] Security upgrade wheel from 0.30.0 to 0.38.0

    This PR was automatically created by Snyk using the credentials of a real user.


    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt
    ⚠️ Warning 
    torchvision 0.5.0 requires pillow, which is not installed.
tensorflow 1.14.0 requires grpcio, which is not installed.
tensorflow 1.14.0 requires protobuf, which is not installed.
tensorboard 1.14.0 requires protobuf, which is not installed.
tensorboard 1.14.0 requires grpcio, which is not installed.
parameter-sherpa 1.0.6 requires pymongo, which is not installed.
parameter-sherpa 1.0.6 requires GPyOpt, which is not installed.

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- medium severity | 551/1000
    Why? Recently disclosed, Has a fix available, CVSS 5.3 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-WHEEL-3092128 | wheel:
    0.30.0 -> 0.38.0
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    Learn how to fix vulnerabilities with free interactive lessons:

    🦉 Regular Expression Denial of Service (ReDoS)

    opened by Emekaborisama 0
  • [Snyk] Fix for 23 vulnerabilities

    [Snyk] Fix for 23 vulnerabilities

    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt
    ⚠️ Warning 
    torchvision 0.5.0 requires pillow, which is not installed.
parameter-sherpa 1.0.6 requires scikit-learn, which is not installed.
GPy 1.10.0 requires paramz, which is not installed.
GPy 1.10.0 requires cython, which is not installed.
GPy 1.10.0 has requirement scipy<1.5.0,>=1.3.0, but you have scipy 1.2.3.

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1055461 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1055462 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 509/1000
    Why? Has a fix available, CVSS 5.9 | Out-of-bounds Write
    SNYK-PYTHON-PILLOW-1059090 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1080635 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-PILLOW-1080654 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1081494 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1081501 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1081502 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 654/1000
    Why? Has a fix available, CVSS 8.8 | Heap-based Buffer Overflow
    SNYK-PYTHON-PILLOW-1082329 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Insufficient Validation
    SNYK-PYTHON-PILLOW-1082750 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1090584 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1090586 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1090587 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Denial of Service (DoS)
    SNYK-PYTHON-PILLOW-1090588 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1292150 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit high severity | 589/1000
    Why? Has a fix available, CVSS 7.5 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-1292151 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 566/1000
    Why? Recently disclosed, Has a fix available, CVSS 5.6 | Buffer Overflow
    SNYK-PYTHON-PILLOW-1316216 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 414/1000
    Why? Has a fix available, CVSS 4 | Out-of-Bounds
    SNYK-PYTHON-PILLOW-574573 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 414/1000
    Why? Has a fix available, CVSS 4 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-574574 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 414/1000
    Why? Has a fix available, CVSS 4 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-574575 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 414/1000
    Why? Has a fix available, CVSS 4 | Out-of-bounds Read
    SNYK-PYTHON-PILLOW-574576 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit medium severity | 469/1000
    Why? Has a fix available, CVSS 5.1 | Buffer Overflow
    SNYK-PYTHON-PILLOW-574577 | pillow:
    6.2.2 -> 8.1.0
    | No | No Known Exploit low severity | 506/1000
    Why? Proof of Concept exploit, Has a fix available, CVSS 3.7 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-SCIKITLEARN-1079100 | scikit-learn:
    0.20.4 -> 0.24.2
    | No | Proof of Concept

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.

    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    opened by snyk-bot 0
Releases(v1.9.0)
Owner
Emeka boris ama
Machine Learning Engineer, Data Scientist, Youtuber and Advocacy
Emeka boris ama
GitHub Repository
Graphical user interface for Argos Translate

Argos Translate GUI Website | GitHub | PyPI Graphical user interface for Argos Translate. Install pip3 install argostranslategui

Argos Open Tech 16 Dec 07, 2022
Rich Prosody Diversity Modelling with Phone-level Mixture Density Network

Phone Level Mixture Density Network for TTS This repo contains pytorch implementation of paper Rich Prosody Diversity Modelling with Phone-level Mixtu

Rishikesh (ऋषिकेश) 42 Dec 13, 2022
Final Project for the Intel AI Readiness Boot Camp NLP (Jan)

NLP Boot Camp (Jan) Synopsis Full Name: Prameya Mohanty Name of your School: Delhi Public School, Rourkela Class: VIII Title of the Project: iTransect

TheCodingHub 1 Feb 01, 2022
vits chinese, tts chinese, tts mandarin

vits chinese, tts chinese, tts mandarin 史上训练最简单，音质最好的语音合成系统

AmorTX 12 Dec 14, 2022
Google's Meena transformer chatbot implementation

Here's my attempt at recreating Meena, a state of the art chatbot developed by Google Research and described in the paper Towards a Human-like Open-Domain Chatbot.

Francesco Pham 94 Dec 25, 2022
This repository collects together basic linguistic processing data for using dataset dumps from the Common Voice project

Common Voice Utils This repository collects together basic linguistic processing data for using dataset dumps from the Common Voice project. It aims t

Francis Tyers 40 Dec 20, 2022
Natural Language Processing for Adverse Drug Reaction (ADR) Detection

Natural Language Processing for Adverse Drug Reaction (ADR) Detection This repo contains code from a project to identify ADRs in discharge summaries a

Medicines Optimisation Service - Austin Health 21 Aug 05, 2022
Kestrel Threat Hunting Language

Kestrel Threat Hunting Language What is Kestrel? Why we need it? How to hunt with XDR support? What is the science behind it? You can find all the ans

Open Cybersecurity Alliance 201 Dec 16, 2022
gaiic2021-track3-小布助手对话短文本语义匹配复赛rank3、决赛rank4

决赛答辩已经过去一段时间了，我们队伍ac milan最终获得了复赛第3，决赛第4的成绩。在此首先感谢一些队友的carry～ 经过2个多月的比赛，学习收获了很多，也认识了很多大佬，在这里记录一下自己的参赛体验和学习收获。

102 Dec 19, 2022
Production First and Production Ready End-to-End Keyword Spotting Toolkit

Production First and Production Ready End-to-End Keyword Spotting Toolkit

223 Jan 02, 2023
中文医疗信息处理基准CBLUE: A Chinese Biomedical LanguageUnderstanding Evaluation Benchmark

English | 中文说明 CBLUE AI (Artificial Intelligence) is playing an indispensabe role in the biomedical field, helping improve medical technology. For fur

452 Dec 30, 2022
A Chinese to English Neural Model Translation Project

ZH-EN NMT Chinese to English Neural Machine Translation This project is inspired by Stanford's CS224N NMT Project Dataset used in this project: News C

Zhenbang Feng 29 Nov 26, 2022
CodeBERT: A Pre-Trained Model for Programming and Natural Languages.

CodeBERT This repo provides the code for reproducing the experiments in CodeBERT: A Pre-Trained Model for Programming and Natural Languages. CodeBERT

Microsoft 1k Jan 03, 2023
Neural building blocks for speaker diarization: speech activity detection, speaker change detection, overlapped speech detection, speaker embedding

⚠️ Checkout develop branch to see what is coming in pyannote.audio 2.0: a much smaller and cleaner codebase Python-first API (the good old pyannote-au

pyannote 2.2k Jan 09, 2023
RoNER is a Named Entity Recognition model based on a pre-trained BERT transformer model trained on RONECv2

RoNER RoNER is a Named Entity Recognition model based on a pre-trained BERT transformer model trained on RONECv2. It is meant to be an easy to use, hi

Stefan Dumitrescu 9 Nov 07, 2022
Perform sentiment analysis and keyword extraction on Craigslist listings

craiglist-helper synopsis Perform sentiment analysis and keyword extraction on Craigslist listings Background I love Craigslist. I've found most of my

Mark Musil 1 Nov 08, 2021
基于pytorch_rnn的古诗词生成

pytorch_peot_rnn 基于pytorch_rnn的古诗词生成 说明 config.py里面含有训练、测试、预测的参数，更改后运行： python main.py 预测结果 if config.do_predict: result = trainer.generate('丽日照残春')

西西嘛呦 3 May 26, 2022
Basic Utilities for PyTorch Natural Language Processing (NLP)

Basic Utilities for PyTorch Natural Language Processing (NLP) PyTorch-NLP, or torchnlp for short, is a library of basic utilities for PyTorch NLP. tor

Michael Petrochuk 2.1k Jan 01, 2023
A relatively simple python program to generate one of those reddit text to speech videos dominating youtube.

Reddit text to speech generator A basic reddit tts video generator Current functionality Generate videos for subs based on comments,(askreddit) so rea

Aadvik 17 Dec 19, 2022
Py65 65816 - Add support for the 65C816 to py65

Add support for the 65C816 to py65 Py65 (https://github.com/mnaberez/py65) is a

4 Jan 04, 2023