RITA is a family of autoregressive protein models, developed by LightOn in collaboration with the OATML group at Oxford and the Debora Marks Lab at Harvard.

Last update: Dec 22, 2022

Overview

RITA: a Study on Scaling Up Generative Protein Sequence Models

RITA is a family of autoregressive protein models, developed by a collaboration of Lighton, the OATML group at Oxford, and the Debbie Marks Lab at Harvard.

Model	#Params	d_model	layers	lm loss uniref-100
Small	85M	768	12	2.31
Medium	300M	1024	24	2.01
Large	680M	1536	24	1.82
XLarge	1.2B	2048	24	1.70

Results

For full results see our preprint: https://arxiv.org/abs/2205.05789

Usage

Instantiate a model like so:

from transformers import AutoModel, AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("lightonai/RITA_s, trust_remote_code=True")
tokenizer = AutoTokenizer.from_pretrained("lightonai/RITA_s")

for generation we support pipelines:

from transformers import pipeline
rita_gen = pipeline('text-generation', model=model, tokenizer=tokenizer)
sequences = rita_gen("MAB", max_length=20, do_sample=True, top_k=950, repetition_penalty=1.2, 
                     num_return_sequences=2, eos_token_id=2)
for seq in sequences:
    print(f"seq: {seq['generated_text'].replace(' ', '')}")

Or see example.py

How to cite

@article{hesslow2022rita,
  title={RITA: a Study on Scaling Up Generative Protein Sequence Models},
  author={Hesslow, Daniel and Zanichelli, Niccol{\'o} and Notin, Pascal and Poli, Iacopo and Marks, Debora},
  journal={arXiv preprint arXiv:2205.05789},
  year={2022}
}

RITA is a family of autoregressive protein models, developed by LightOn in collaboration with the OATML group at Oxford and the Debora Marks Lab at Harvard.

Related tags

Overview

RITA: a Study on Scaling Up Generative Protein Sequence Models

Results

Usage

How to cite

Owner

LightOn

A distributed, plug-n-play algorithm for multi-robot applications with a priori non-computable objective functions

Unofficial implementation of HiFi-GAN+ from the paper "Bandwidth Extension is All You Need" by Su, et al.

A production-ready, scalable Indexer for the Jina neural search framework, based on HNSW and PSQL

PyTorch Code for "Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning"

Unofficial PyTorch Implementation of "DOLG: Single-Stage Image Retrieval with Deep Orthogonal Fusion of Local and Global Features"

ByteTrack(Multi-Object Tracking by Associating Every Detection Box)のPythonでのONNX推論サンプル

PyTorch Lightning implementation of Automatic Speech Recognition

GPU implementation of $k$-Nearest Neighbors and Shared-Nearest Neighbors

SalGAN: Visual Saliency Prediction with Generative Adversarial Networks

Spherical CNNs

Code for DeepCurrents: Learning Implicit Representations of Shapes with Boundaries

Datasets, tools, and benchmarks for representation learning of code.

Face-Recognition-Attendence-System - This face recognition Attendence system using Python

A PyTorch implementation of "TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?"

An End-to-End Machine Learning Library to Optimize AUC (AUROC, AUPRC).

Instantaneous Motion Generation for Robots and Machines.

Definition of a business problem according to Wilson Lower Bound Score and Time Based Average Rating

A Broader Picture of Random-walk Based Graph Embedding

Reimplementation of NeurIPS'19: "Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting" by Shu et al.

Simple helper library to convert a collection of numpy data to tfrecord, and build a tensorflow dataset from the tfrecord.