banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services.

Last update: Dec 22, 2022

Overview

What's banditml?

banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services. This library is developed by Bandit ML and ex-authors of Facebook's applied reinforcement learning platform, Reagent.

Specifically, this repo contains:

Feature engineering & preprocessing
Model implementations
Model training workflows
Model serving code for Python services

Supported models

Models supported:

Contextual Bandits (small datasets)
- Linear bandit w/ ε-greedy exploration
- Random forest bandit w/ ε-greedy exploration
- Gradient boosted decision tree bandit w/ ε-greedy exploration
Contextual Bandits (medium datasets)
- Neural bandit with ε-greedy exploration
- Neural bandit with UCB-based exploration (via. dropout exploration)
- Neural bandit with UCB-based exploration (via. mixture density networks)
Reinforcement Learning (large datasets)

4 feature types supported:

Numeric: standard floating point features
- e.g. {totalCartValue: 39.99}
Categorical: low-cardinality discrete features
- e.g. {currentlyViewingCategory: "men's jeans"}
ID list: high-cardinality discrete features
- e.g. {productsInCart: ["productId022", "productId109"...]}
- Handled via. learned embedding tables
"Dense" ID list: high-cardinality discrete features, manually mapped to dense feature vectors
- e.g {productId022: [0.5, 1.3, ...], productId109: [1.9, 0.1, ...], ...}

Docs

pip install banditml

Get started

License

GNU General Public License v3.0 or later

See COPYING to see the full text.

You might also like...

Repo for CVPR2021 paper "QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information"

QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information by Masato Tamura, Hiroki Ohashi, and Tomoaki Yosh

105 Dec 23, 2022

Source code and data from the RecSys 2020 article "Carousel Personalization in Music Streaming Apps with Contextual Bandits" by W. Bendada, G. Salha and T. Bontempelli

Carousel Personalization in Music Streaming Apps with Contextual Bandits - RecSys 2020 This repository provides Python code and data to reproduce expe

48 Jan 2, 2023

UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus

UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus General info This is

71 Oct 25, 2022

Generate Contextual Directory Wordlist For Target Org

PathPermutor Generate Contextual Directory Wordlist For Target Org This script generates contextual wordlist for any target org based on the set of UR

8 Jun 23, 2021

ICCV2021 - Mining Contextual Information Beyond Image for Semantic Segmentation

Introduction The official repository for "Mining Contextual Information Beyond Image for Semantic Segmentation". Our full code has been merged into ss

55 Nov 9, 2022

[2021 MultiMedia] CONQUER: Contextual Query-aware Ranking for Video Corpus Moment Retrieval

CONQUER: Contexutal Query-aware Ranking for Video Corpus Moment Retreival PyTorch implementation of CONQUER: Contexutal Query-aware Ranking for Video

23 Dec 26, 2022

Official implementation of NeurIPS 2021 paper "Contextual Similarity Aggregation with Self-attention for Visual Re-ranking"

CSA: Contextual Similarity Aggregation with Self-attention for Visual Re-ranking PyTorch training code for CSA (Contextual Similarity Aggregation). We

19 Oct 21, 2022

Official Pytorch implementation for Deep Contextual Video Compression, NeurIPS 2021

Introduction Official Pytorch implementation for Deep Contextual Video Compression, NeurIPS 2021 Prerequisites Python 3.8 and conda, get Conda CUDA 11

51 Dec 3, 2022

Code and data for ImageCoDe, a contextual vison-and-language benchmark

ImageCoDe This repository contains code and data for ImageCoDe: Image Retrieval from Contextual Descriptions. Data All collected descriptions for the

27 Dec 2, 2022

Comments

Adapting ABTest data to contextual bandit setting

Hi, and thanks for open sourcing this project.

I wanted to dive into it by testing some ABTesting data with the implemented neural bandit.

In my setting I have only 2 choices, 121 features as context, a reward range of [0.0, 120], and only 11% rows have non-zero reward. After training for a few epoch I see the testing loss decreasing a bit. But at test time, scores of the two choices are always equals, and the ucb_scores always equal to 0.

opened by virgile-blg 0
Model input dimension does not update when keeping top n features

Setting : Neural Bandit

When setting keep_only_top_n to True, the model keeps the original number of features, resulting in a Pytorch matmul error for the first linear layer:

RuntimeError: mat1 and mat2 shapes cannot be multiplied (256x10 and 121x64)

opened by virgile-blg 0

banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services.

Related tags

Overview

What's banditml?

Supported models

Docs

License

You might also like...

Repo for CVPR2021 paper "QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information"

Source code and data from the RecSys 2020 article "Carousel Personalization in Music Streaming Apps with Contextual Bandits" by W. Bendada, G. Salha and T. Bontempelli

UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus

Generate Contextual Directory Wordlist For Target Org

ICCV2021 - Mining Contextual Information Beyond Image for Semantic Segmentation

[2021 MultiMedia] CONQUER: Contextual Query-aware Ranking for Video Corpus Moment Retrieval

Official implementation of NeurIPS 2021 paper "Contextual Similarity Aggregation with Self-attention for Visual Re-ranking"

Official Pytorch implementation for Deep Contextual Video Compression, NeurIPS 2021

Code and data for ImageCoDe, a contextual vison-and-language benchmark

Comments

Adapting ABTest data to contextual bandit setting

Model input dimension does not update when keeping top n features

Releases(1.0.2)

1.0.2(Jun 4, 2021)

Owner

Bandit ML

This is a code repository for paper OODformer: Out-Of-Distribution Detection Transformer

Training Cifar-10 Classifier Using VGG16

CenterNet:Objects as Points目标检测模型在Pytorch当中的实现

Predicting lncRNA–protein interactions based on graph autoencoders and collaborative training

[CVPR 2021] MetaSAug: Meta Semantic Augmentation for Long-Tailed Visual Recognition

Code accompanying "Adaptive Methods for Aggregated Domain Generalization"

Directed Greybox Fuzzing with AFL

Codes for the compilation and visualization examples to the HIF vegetation dataset

Machine learning notebooks in different subjects optimized to run in google collaboratory

Tensorflow solution of NER task Using BiLSTM-CRF model with Google BERT Fine-tuning And private Server services

Code for One-shot Talking Face Generation from Single-speaker Audio-Visual Correlation Learning (AAAI 2022)

Groceries ARL: Association Rules (Birliktelik Kuralı)

本项目是一个带有前端界面的垃圾分类项目，加载了训练好的模型参数，模型为efficientnetb4，暂时为40分类问题。

Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks

Source code for our paper "Molecular Mechanics-Driven Graph Neural Network with Multiplex Graph for Molecular Structures"

JstDoS - HTTP Protocol Stack Remote Code Execution Vulnerability

Re-TACRED: Addressing Shortcomings of the TACRED Dataset

Implementation of a Transformer that Ponders, using the scheme from the PonderNet paper

Deep Reinforcement Learning with pytorch & visdom

MinkLoc3D-SI: 3D LiDAR place recognition with sparse convolutions,spherical coordinates, and intensity