RITA is a family of autoregressive protein models, developed by LightOn in collaboration with the OATML group at Oxford and the Debora Marks Lab at Harvard.

Last update: Dec 22, 2022

Overview

RITA: a Study on Scaling Up Generative Protein Sequence Models

RITA is a family of autoregressive protein models, developed by a collaboration of Lighton, the OATML group at Oxford, and the Debbie Marks Lab at Harvard.

Model	#Params	d_model	layers	lm loss uniref-100
Small	85M	768	12	2.31
Medium	300M	1024	24	2.01
Large	680M	1536	24	1.82
XLarge	1.2B	2048	24	1.70

Results

For full results see our preprint: https://arxiv.org/abs/2205.05789

Usage

Instantiate a model like so:

from transformers import AutoModel, AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("lightonai/RITA_s, trust_remote_code=True")
tokenizer = AutoTokenizer.from_pretrained("lightonai/RITA_s")

for generation we support pipelines:

from transformers import pipeline
rita_gen = pipeline('text-generation', model=model, tokenizer=tokenizer)
sequences = rita_gen("MAB", max_length=20, do_sample=True, top_k=950, repetition_penalty=1.2, 
                     num_return_sequences=2, eos_token_id=2)
for seq in sequences:
    print(f"seq: {seq['generated_text'].replace(' ', '')}")

Or see example.py

How to cite

@article{hesslow2022rita,
  title={RITA: a Study on Scaling Up Generative Protein Sequence Models},
  author={Hesslow, Daniel and Zanichelli, Niccol{\'o} and Notin, Pascal and Poli, Iacopo and Marks, Debora},
  journal={arXiv preprint arXiv:2205.05789},
  year={2022}
}

RITA is a family of autoregressive protein models, developed by LightOn in collaboration with the OATML group at Oxford and the Debora Marks Lab at Harvard.

Related tags

Overview

RITA: a Study on Scaling Up Generative Protein Sequence Models

Results

Usage

How to cite

Owner

LightOn

Western-3DSlicer-Modules - Point-Set Registrations for Ultrasound Probe Calibrations

A MatConvNet-based implementation of the Fully-Convolutional Networks for image segmentation

Code of our paper "Contrastive Object-level Pre-training with Spatial Noise Curriculum Learning"

Recommendation algorithms for large graphs

Code for approximate graph reduction techniques for cardinality-based DSFM, from paper

Pytoydl: A toy deep learning framework built upon numpy.

PyTorch3D is FAIR's library of reusable components for deep learning with 3D data

Official implementation for ICDAR 2021 paper "Handwritten Mathematical Expression Recognition with Bidirectionally Trained Transformer"

Complete system for facial identity system. Include one-shot model, database operation, features visualization, monitoring

Multi-Agent Reinforcement Learning (MARL) method to learn scalable control polices for multi-agent target tracking.

This repo is a C++ version of yolov5_deepsort_tensorrt. Packing all C++ programs into .so files, using Python script to call C++ programs further.

A library for uncertainty representation and training in neural networks.

Deep learning library featuring a higher-level API for TensorFlow.

PyTorch Implementation of Region Similarity Representation Learning (ReSim)

A Python package to create, run, and post-process MODFLOW-based models.

A crash course in six episodes for software developers who want to become machine learning practitioners.

Code for the ICCV2021 paper "Personalized Image Semantic Segmentation"

Implementations of polygamma, lgamma, and beta functions for PyTorch

Official implementation of NeurIPS'21: Implicit SVD for Graph Representation Learning

A robotic arm that mimics hand movement through MediaPipe tracking.