An efficient PyTorch implementation of the evaluation metrics in recommender systems.

Last update: Dec 02, 2022

Related tags

Deep Learning recsys_metrics

Overview

recsys_metrics

An efficient PyTorch implementation of the evaluation metrics in recommender systems.

Overview • Installation • How to use • Benchmark • Citation •

Overview

Highlights

Efficient (vectorized) implementations over mini-batches
Standard RecSys metrics: precision, recall, map, mrr, hr, ndcg
Beyond-accuracy metrics: e.g. coverage, diversity, novelty, etc.
All metrics support a top-k argument.

Why do we need `recsys_metrics`?

Installation

You can install recsys_metrics from PyPI:

pip install recsys_metrics

Or you can also install the latest version from source:

pip install git+https://github.com/zuoxingdong/[email protected]

Note that we support Python 3.7+ only.

How to use

Let us take Hit Rate (HR) to illustrate how to use this library:

preds = torch.tensor([
    [.5, .3, .1],
    [.3, .4, .5]
])
target = torch.tensor([
    [0, 0, 1],
    [0, 1, 1]
])
hit_rate(preds, target, k=1, reduction='mean')

>> tensor(0.5000)

The first example in the batch does not have a hit (i.e. top-1 item is not a relevant item) and second example in the batch gets a hit (i.e. top-1 item is a relevant item). Thus, we have a hit-rate of 0.5.

The API of other metrics are of the same format.

Benchmark

Metrics	Single Example	Mini-Batch
Precision
Recall
MAP
MRR
HR
NDCG

Citation

This work is inspired by Torchmetrics from PyTorchLightning Team.

Please use this bibtex if you want to cite this repository in your publications:

@misc{recsys_metrics,
      author = {Zuo, Xingdong},
      title = {recsys_metrics: An efficient PyTorch implementation of the evaluation metrics in recommender systems.},
      year = {2021},
      publisher = {GitHub},
      journal = {GitHub repository},
      howpublished = {\url{https://github.com/zuoxingdong/recsys_metrics}},
    }

NVIDIA Merlin is an open source library providing end-to-end GPU-accelerated recommender systems, from feature engineering and preprocessing to training deep learning models and running inference in production.

NVIDIA Merlin NVIDIA Merlin is an open source library designed to accelerate recommender systems on NVIDIA’s GPUs. It enables data scientists, machine

419 Jan 3, 2023

This is the official repository for evaluation on the NoW Benchmark Dataset. The goal of the NoW benchmark is to introduce a standard evaluation metric to measure the accuracy and robustness of 3D face reconstruction methods from a single image under variations in viewing angle, lighting, and common occlusions.

NoW Evaluation This is the official repository for evaluation on the NoW Benchmark Dataset. The goal of the NoW benchmark is to introduce a standard e

71 Dec 30, 2022

StackRec: Efficient Training of Very Deep Sequential Recommender Models by Iterative Stacking

StackRec: Efficient Training of Very Deep Sequential Recommender Models by Iterative Stacking Datasets You can download datasets that have been pre-pr

25 May 29, 2022

Active and Sample-Efficient Model Evaluation

Active Testing: Sample-Efficient Model Evaluation Hi, good to see you here! 👋 This is code for "Active Testing: Sample-Efficient Model Evaluation". P

19 Oct 30, 2022

Efficient-GlobalPointer - Pytorch Efficient GlobalPointer

引言感谢苏神带来的模型，原文地址：https://spaces.ac.cn/archives/8877 如何运行对应模型EfficientGlobalPoi

40 Dec 14, 2022

UMEC: Unified Model and Embedding Compression for Efficient Recommendation Systems

[ICLR 2021] "UMEC: Unified Model and Embedding Compression for Efficient Recommendation Systems" by Jiayi Shen, Haotao Wang*, Shupeng Gui*, Jianchao Tan, Zhangyang Wang, and Ji Liu

39 Dec 3, 2022

Crab is a ﬂexible, fast recommender engine for Python that integrates classic information ﬁltering recommendation algorithms in the world of scientiﬁc Python packages (numpy, scipy, matplotlib).

Crab - A Recommendation Engine library for Python Crab is a ﬂexible, fast recommender engine for Python that integrates classic information ﬁltering r

1.2k Dec 21, 2022

A python library for implementing a recommender system

python-recsys A python library for implementing a recommender system. Installation Dependencies python-recsys is build on top of Divisi2, with csc-pys

1.5k Dec 17, 2022

A TikTok-like recommender system for GitHub repositories based on Gorse

GitRec GitRec is the missing recommender system for GitHub repositories based on Gorse. Architecture The trending crawler crawls trending repositories

337 Jan 4, 2023

An efficient PyTorch implementation of the evaluation metrics in recommender systems.

Related tags

Overview

Overview

Highlights

Why do we need recsys_metrics?

Installation

How to use

Benchmark

Citation

You might also like...

NVIDIA Merlin is an open source library providing end-to-end GPU-accelerated recommender systems, from feature engineering and preprocessing to training deep learning models and running inference in production.

StackRec: Efficient Training of Very Deep Sequential Recommender Models by Iterative Stacking

Active and Sample-Efficient Model Evaluation

Efficient-GlobalPointer - Pytorch Efficient GlobalPointer

UMEC: Unified Model and Embedding Compression for Efficient Recommendation Systems

Crab is a ﬂexible, fast recommender engine for Python that integrates classic information ﬁltering recommendation algorithms in the world of scientiﬁc Python packages (numpy, scipy, matplotlib).

A python library for implementing a recommender system

A TikTok-like recommender system for GitHub repositories based on Gorse

Releases(v0.0.3)

v0.0.3(Aug 1, 2022)

v0.0.2(Jun 14, 2022)

Owner

Xingdong Zuo

Enabling dynamic analysis of Legacy Embedded Systems in full emulated environment

PyTorch implementations of the paper: "Learning Independent Instance Maps for Crowd Localization"

Pytorch reimplementation of the Mixer (MLP-Mixer: An all-MLP Architecture for Vision)

Optical Character Recognition + Instance Segmentation for russian and english languages

JFB: Jacobian-Free Backpropagation for Implicit Models

This project generates news headlines using a Long Short-Term Memory (LSTM) neural network.

Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations. [2021]

IA for recognising Traffic Signs using Keras [Tensorflow]

Local Multi-Head Channel Self-Attention for FER2013

Luminaire is a python package that provides ML driven solutions for monitoring time series data.

[NeurIPS 2021] COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining

DziriBERT: a Pre-trained Language Model for the Algerian Dialect

机器学习、深度学习、自然语言处理等人工智能基础知识总结。

基于tensorflow 2.x的图片识别工具集

CBKH: The Cornell Biomedical Knowledge Hub

Neural Turing Machines (NTM) - PyTorch Implementation

Cards Against Humanity AI

SSD-based Object Detection in PyTorch

TensorFlow implementation of Barlow Twins (Barlow Twins: Self-Supervised Learning via Redundancy Reduction)

Code for our EMNLP 2021 paper "Learning Kernel-Smoothed Machine Translation with Retrieved Examples"

Why do we need `recsys_metrics`?