banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services.

Last update: Dec 22, 2022

Overview

What's banditml?

banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services. This library is developed by Bandit ML and ex-authors of Facebook's applied reinforcement learning platform, Reagent.

Specifically, this repo contains:

Feature engineering & preprocessing
Model implementations
Model training workflows
Model serving code for Python services

Supported models

Models supported:

Contextual Bandits (small datasets)
- Linear bandit w/ ε-greedy exploration
- Random forest bandit w/ ε-greedy exploration
- Gradient boosted decision tree bandit w/ ε-greedy exploration
Contextual Bandits (medium datasets)
- Neural bandit with ε-greedy exploration
- Neural bandit with UCB-based exploration (via. dropout exploration)
- Neural bandit with UCB-based exploration (via. mixture density networks)
Reinforcement Learning (large datasets)

4 feature types supported:

Numeric: standard floating point features
- e.g. {totalCartValue: 39.99}
Categorical: low-cardinality discrete features
- e.g. {currentlyViewingCategory: "men's jeans"}
ID list: high-cardinality discrete features
- e.g. {productsInCart: ["productId022", "productId109"...]}
- Handled via. learned embedding tables
"Dense" ID list: high-cardinality discrete features, manually mapped to dense feature vectors
- e.g {productId022: [0.5, 1.3, ...], productId109: [1.9, 0.1, ...], ...}

Docs

pip install banditml

Get started

License

GNU General Public License v3.0 or later

See COPYING to see the full text.

You might also like...

Repo for CVPR2021 paper "QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information"

QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information by Masato Tamura, Hiroki Ohashi, and Tomoaki Yosh

105 Dec 23, 2022

Source code and data from the RecSys 2020 article "Carousel Personalization in Music Streaming Apps with Contextual Bandits" by W. Bendada, G. Salha and T. Bontempelli

Carousel Personalization in Music Streaming Apps with Contextual Bandits - RecSys 2020 This repository provides Python code and data to reproduce expe

48 Jan 2, 2023

UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus

UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus General info This is

71 Oct 25, 2022

Generate Contextual Directory Wordlist For Target Org

PathPermutor Generate Contextual Directory Wordlist For Target Org This script generates contextual wordlist for any target org based on the set of UR

8 Jun 23, 2021

ICCV2021 - Mining Contextual Information Beyond Image for Semantic Segmentation

Introduction The official repository for "Mining Contextual Information Beyond Image for Semantic Segmentation". Our full code has been merged into ss

55 Nov 9, 2022

[2021 MultiMedia] CONQUER: Contextual Query-aware Ranking for Video Corpus Moment Retrieval

CONQUER: Contexutal Query-aware Ranking for Video Corpus Moment Retreival PyTorch implementation of CONQUER: Contexutal Query-aware Ranking for Video

23 Dec 26, 2022

Official implementation of NeurIPS 2021 paper "Contextual Similarity Aggregation with Self-attention for Visual Re-ranking"

CSA: Contextual Similarity Aggregation with Self-attention for Visual Re-ranking PyTorch training code for CSA (Contextual Similarity Aggregation). We

19 Oct 21, 2022

Official Pytorch implementation for Deep Contextual Video Compression, NeurIPS 2021

Introduction Official Pytorch implementation for Deep Contextual Video Compression, NeurIPS 2021 Prerequisites Python 3.8 and conda, get Conda CUDA 11

51 Dec 3, 2022

Code and data for ImageCoDe, a contextual vison-and-language benchmark

ImageCoDe This repository contains code and data for ImageCoDe: Image Retrieval from Contextual Descriptions. Data All collected descriptions for the

27 Dec 2, 2022

Comments

Adapting ABTest data to contextual bandit setting

Hi, and thanks for open sourcing this project.

I wanted to dive into it by testing some ABTesting data with the implemented neural bandit.

In my setting I have only 2 choices, 121 features as context, a reward range of [0.0, 120], and only 11% rows have non-zero reward. After training for a few epoch I see the testing loss decreasing a bit. But at test time, scores of the two choices are always equals, and the ucb_scores always equal to 0.

opened by virgile-blg 0
Model input dimension does not update when keeping top n features

Setting : Neural Bandit

When setting keep_only_top_n to True, the model keeps the original number of features, resulting in a Pytorch matmul error for the first linear layer:

RuntimeError: mat1 and mat2 shapes cannot be multiplied (256x10 and 121x64)

opened by virgile-blg 0

banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services.

Related tags

Overview

What's banditml?

Supported models

Docs

License

You might also like...

Repo for CVPR2021 paper "QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information"

Source code and data from the RecSys 2020 article "Carousel Personalization in Music Streaming Apps with Contextual Bandits" by W. Bendada, G. Salha and T. Bontempelli

UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus

Generate Contextual Directory Wordlist For Target Org

ICCV2021 - Mining Contextual Information Beyond Image for Semantic Segmentation

[2021 MultiMedia] CONQUER: Contextual Query-aware Ranking for Video Corpus Moment Retrieval

Official implementation of NeurIPS 2021 paper "Contextual Similarity Aggregation with Self-attention for Visual Re-ranking"

Official Pytorch implementation for Deep Contextual Video Compression, NeurIPS 2021

Code and data for ImageCoDe, a contextual vison-and-language benchmark

Comments

Adapting ABTest data to contextual bandit setting

Model input dimension does not update when keeping top n features

Releases(1.0.2)

1.0.2(Jun 4, 2021)

Owner

Bandit ML

Fine-tuning StyleGAN2 for Cartoon Face Generation

Xview3 solution - XView3 challenge, 2nd place solution

FLAVR is a fast, flow-free frame interpolation method capable of single shot multi-frame prediction

Example how to deploy deep learning model with aiohttp.

Unofficial PyTorch implementation of Guided Dropout

Repository to run object detection on a model trained on an autonomous driving dataset.

Official Code Implementation of the paper : XAI for Transformers: Better Explanations through Conservative Propagation

A very short and easy implementation of Quantile Regression DQN

Official repository for Hierarchical Opacity Propagation for Image Matting

An open-source Kazakh named entity recognition dataset (KazNERD), annotation guidelines, and baseline NER models.

Decoding the Protein-ligand Interactions Using Parallel Graph Neural Networks

Official code of the paper "Expanding Low-Density Latent Regions for Open-Set Object Detection" (CVPR 2022)

official Pytorch implementation of ICCV 2021 paper FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting.

ONNX-GLPDepth - Python scripts for performing monocular depth estimation using the GLPDepth model in ONNX

You Only Hypothesize Once: Point Cloud Registration with Rotation-equivariant Descriptors

Tools to create pixel-wise object masks, bounding box labels (2D and 3D) and 3D object model (PLY triangle mesh) for object sequences filmed with an RGB-D camera.

Supervised forecasting of sequential data in Python.

DCT-Mask: Discrete Cosine Transform Mask Representation for Instance Segmentation

Pytorch Implementation of Auto-Compressing Subset Pruning for Semantic Image Segmentation

This repository contains the data and code for the paper "Diverse Text Generation via Variational Encoder-Decoder Models with Gaussian Process Priors" ([email protected])