Simple Annotated implementation of GPT-NeoX in PyTorch
This is a simpler implementation of GPT-NeoX in PyTorch. We have taken out several optimizations in GPT-NeoX for simplicity.
Annotated implementation
Simple Annotated implementation of GPT-NeoX in PyTorch
Overview
Owner
labml.ai
Tools to help deep learning researchers
Honor's thesis project analyzing whether the GPT-2 model can more effectively generate free-verse or structured poetry.
gpt2-poetry The following code is for my senior honor's thesis project, under the guidance of Dr. Keith Holyoak at the University of California, Los A
2 Jan 09, 2022
This codebase facilitates fast experimentation of differentially private training of Hugging Face transformers.
private-transformers This codebase facilitates fast experimentation of differentially private training of Hugging Face transformers. What is this? Why
73 Dec 28, 2022
Creating a chess engine using GPT-3
GPT3Chess Creating a chess engine using GPT-3 Code for my article : https://towardsdatascience.com/gpt-3-play-chess-d123a96096a9 My game (white) vs GP
19 Dec 17, 2022
Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding
Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding
61 Dec 21, 2022
Research code for "What to Pre-Train on? Efficient Intermediate Task Selection", EMNLP 2021
efficient-task-transfer This repository contains code for the experiments in our paper "What to Pre-Train on? Efficient Intermediate Task Selection".
26 Dec 24, 2022
Unofficial implementation of Google's FNet: Mixing Tokens with Fourier Transforms
FNet: Mixing Tokens with Fourier Transforms Pytorch implementation of Fnet : Mixing Tokens with Fourier Transforms. Citation: @misc{leethorp2021fnet,
217 Dec 05, 2022
A full spaCy pipeline and models for scientific/biomedical documents.
This repository contains custom pipes and models related to using spaCy for scientific documents. In particular, there is a custom tokenizer that adds
1.3k Jan 03, 2023
The code for two papers: Feedback Transformer and Expire-Span.
transformer-sequential This repo contains the code for two papers: Feedback Transformer Expire-Span The training code is structured for long sequentia
125 Dec 25, 2022
Simple Annotated implementation of GPT-NeoX in PyTorch
Simple Annotated implementation of GPT-NeoX in PyTorch This is a simpler implementation of GPT-NeoX in PyTorch. We have taken out several optimization
101 Dec 03, 2022
Creating an LSTM model to generate music
Music-Generation Creating an LSTM model to generate music music-generator Used to create basic sin wave sounds music-ai Contains the functions to conv
2 Dec 02, 2021
DeepPavlov Tutorials
DeepPavlov tutorials DeepPavlov: Sentence Classification with Word Embeddings DeepPavlov: Transfer Learning with BERT. Classification, Tagging, QA, Ze
28 Sep 13, 2022
An open-source NLP library: fast text cleaning and preprocessing.
An open-source NLP library: fast text cleaning and preprocessing
21 Mar 18, 2022
Pretrain CPM - 大规模预训练语言模型的预训练代码
CPM-Pretrain 版本更新记录 为了促进中文自然语言处理研究的发展，本项目提供了大规模预训练语言模型的预训练代码。项目主要基于DeepSpeed、Megatron实现，可以支持数据并行、模型加速、流水并行的代码。 安装 1、首先安装pytorch等基础依赖，再安装APEX以支持fp16。 p
37 Dec 06, 2022
Use the state-of-the-art m2m100 to translate large data on CPU/GPU/TPU. Super Easy!
Easy-Translate is a script for translating large text files in your machine using the M2M100 models from Facebook/Meta AI. We also privide a script fo
41 Dec 15, 2022
Unsupervised Language Model Pre-training for French
FlauBERT and FLUE FlauBERT is a French BERT trained on a very large and heterogeneous French corpus. Models of different sizes are trained using the n
212 Dec 10, 2022
Tool to check whether a GCP bucket is public or not.
Tool to check publicly accessible GCP bucket. Blog https://justm0rph3u5.medium.com/gcp-inspector-auditing-publicly-exposed-gcp-bucket-ac6cad55618c Wha
7 Nov 24, 2022
[Preprint] Escaping the Big Data Paradigm with Compact Transformers, 2021
Compact Transformers Preprint Link: Escaping the Big Data Paradigm with Compact Transformers By Ali Hassani[1]*, Steven Walton[1]*, Nikhil Shah[1], Ab
367 Dec 31, 2022
Scikit-learn style model finetuning for NLP
Scikit-learn style model finetuning for NLP Finetune is a library that allows users to leverage state-of-the-art pretrained NLP models for a wide vari
665 Dec 17, 2022
Shellcode antivirus evasion framework
Schrodinger's Cat Schrodinger'sCat is a Shellcode antivirus evasion framework Technical principle Please visit my blog https://idiotc4t.com/ How to us
27 Jul 09, 2022
Research code for the paper "Fine-tuning wav2vec2 for speaker recognition"
Fine-tuning wav2vec2 for speaker recognition This is the code used to run the experiments in https://arxiv.org/abs/2109.15053. Detailed logs of each t
103 Dec 26, 2022