MINERVA: An out-of-the-box GUI tool for offline deep reinforcement learning

Overview

MINERVA: An out-of-the-box GUI tool for offline deep reinforcement learning

PyPI version test Docker Cloud Build Status Documentation Status Maintainability codecov MIT

MINERVA is an out-of-the-box GUI tool for offline deep reinforcement learning, designed for everyone including non-programmers to do reinforcement learning as a tool.

Documentation: https://minerva-ui.readthedocs.io

Chat: Gitter

key features

⚡ All You Need Is Dataset

MINERVA only requires datasets to start offline deep reinforcement learning. Any combinations of vector observations and image observations with discrete actions and continuous actions are supported.

đź”° Stunning GUI

MINERVA provides designed with intuitive GUI to let everyone lerverage extremely powerful algorithms without barriers. The GUI is developed as a Single Page Application (SPA) to make it work in the eye-opening speed.

🚀 Powerful Algorithm

MINERVA is powered by d3rlpy, a powerful offline deep reinforcement learning library for Python, to provide extremely powerful algorithms in an out-of-the-box way. The trained policy can be exported as TorchScript and ONNX.

installation

PyPI

$ pip install minerva-ui

Docker

$ docker run -d --gpus all -p 9000:9000 --name minerva takuseno/minerva:latest

update guide

If you update MINERVA, the database schema should be also updated as follows:

$ pip install -U minerva-ui
$ minerva upgrade-db

usage

run server

At the first time, ~/.minerva will be automatically created to store database, uploaded datasets and training metrics.

$ minerva run

By default, you can access to MINERVA interface at http://localhost:9000 . You can change the host and port with --host and --port arguments respectively.

delete data

You can delete entire data (~/.minerva) as follows:

$ minerva clean

contributions

build

$ npm install
$ npm run build

coding style

This repository is fully formatted with yapf and standard. You can format the entire scripts as follows:

$ ./scripts/format

lint

This repository is fully analyzed with Pylint, ESLint and sass-lint. You can run analysis as follows:

$ ./scripts/lint

test

The unit tests are provided as much as possible. This repository is using pytest-cov instead of pytest. You can run the entire tests as follows:

$ ./scripts/test

acknowledgement

This work is supported by Information-technology Promotion Agency, Japan (IPA), Exploratory IT Human Resources Project (MITOU Program) in the fiscal year 2020.

The concept of the GUI software for deep reinforcement learning is inspired by DeepAnalyzer from Ghelia inc. I'm showing the great respect to the team here.

Comments
  • Bump y18n from 4.0.0 to 4.0.3

    Bump y18n from 4.0.0 to 4.0.3

    Bumps y18n from 4.0.0 to 4.0.3.

    Changelog

    Sourced from y18n's changelog.

    4.0.3 (2021-04-07)

    Bug Fixes

    • release: 4.x.x should not enforce Node 10 (#126) (1e21a53)

    4.0.1 (2020-11-30)

    Bug Fixes

    Commits
    Maintainer changes

    This version was pushed to npm by oss-bot, a new releaser for y18n since your current version.


    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies javascript 
    opened by dependabot[bot] 1
  • Bump pillow from 7.1.2 to 8.1.1

    Bump pillow from 7.1.2 to 8.1.1

    Bumps pillow from 7.1.2 to 8.1.1.

    Release notes

    Sourced from pillow's releases.

    8.1.1

    https://pillow.readthedocs.io/en/stable/releasenotes/8.1.1.html

    8.1.0

    https://pillow.readthedocs.io/en/stable/releasenotes/8.1.0.html

    Changes

    Dependencies

    Deprecations

    ... (truncated)

    Changelog

    Sourced from pillow's changelog.

    8.1.1 (2021-03-01)

    • Use more specific regex chars to prevent ReDoS. CVE-2021-25292 [hugovk]

    • Fix OOB Read in TiffDecode.c, and check the tile validity before reading. CVE-2021-25291 [wiredfool]

    • Fix negative size read in TiffDecode.c. CVE-2021-25290 [wiredfool]

    • Fix OOB read in SgiRleDecode.c. CVE-2021-25293 [wiredfool]

    • Incorrect error code checking in TiffDecode.c. CVE-2021-25289 [wiredfool]

    • PyModule_AddObject fix for Python 3.10 #5194 [radarhere]

    8.1.0 (2021-01-02)

    • Fix TIFF OOB Write error. CVE-2020-35654 #5175 [wiredfool]

    • Fix for Read Overflow in PCX Decoding. CVE-2020-35653 #5174 [wiredfool, radarhere]

    • Fix for SGI Decode buffer overrun. CVE-2020-35655 #5173 [wiredfool, radarhere]

    • Fix OOB Read when saving GIF of xsize=1 #5149 [wiredfool]

    • Makefile updates #5159 [wiredfool, radarhere]

    • Add support for PySide6 #5161 [hugovk]

    • Use disposal settings from previous frame in APNG #5126 [radarhere]

    • Added exception explaining that repr_png saves to PNG #5139 [radarhere]

    • Use previous disposal method in GIF load_end #5125 [radarhere]

    ... (truncated)

    Commits
    • 741d874 8.1.1 version bump
    • 179cd1c Added 8.1.1 release notes to index
    • 7d29665 Update CHANGES.rst [ci skip]
    • d25036f Credits
    • 973a4c3 Release notes for 8.1.1
    • 521dab9 Use more specific regex chars to prevent ReDoS
    • 8b8076b Fix for CVE-2021-25291
    • e25be1e Fix negative size read in TiffDecode.c
    • f891baa Fix OOB read in SgiRleDecode.c
    • cbfdde7 Incorrect error code checking in TiffDecode.c
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies python 
    opened by dependabot[bot] 1
  • Bump ini from 1.3.5 to 1.3.8

    Bump ini from 1.3.5 to 1.3.8

    Bumps ini from 1.3.5 to 1.3.8.

    Commits
    • a2c5da8 1.3.8
    • af5c6bb Do not use Object.create(null)
    • 8b648a1 don't test where our devdeps don't even work
    • c74c8af 1.3.7
    • 024b8b5 update deps, add linting
    • 032fbaf Use Object.create(null) to avoid default object property hazards
    • 2da9039 1.3.6
    • cfea636 better git push script, before publish instead of after
    • 56d2805 do not allow invalid hazardous string as section name
    • See full diff in compare view
    Maintainer changes

    This version was pushed to npm by isaacs, a new releaser for ini since your current version.


    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump axios from 0.19.2 to 0.21.1

    Bump axios from 0.19.2 to 0.21.1

    Bumps axios from 0.19.2 to 0.21.1.

    Release notes

    Sourced from axios's releases.

    v0.21.1

    0.21.1 (December 21, 2020)

    Fixes and Functionality:

    • Hotfix: Prevent SSRF (#3410)
    • Protocol not parsed when setting proxy config from env vars (#3070)
    • Updating axios in types to be lower case (#2797)
    • Adding a type guard for AxiosError (#2949)

    Internal and Tests:

    • Remove the skipping of the socket http test (#3364)
    • Use different socket for Win32 test (#3375)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    v0.21.0

    0.21.0 (October 23, 2020)

    Fixes and Functionality:

    • Fixing requestHeaders.Authorization (#3287)
    • Fixing node types (#3237)
    • Fixing axios.delete ignores config.data (#3282)
    • Revert "Fixing overwrite Blob/File type as Content-Type in browser. (#1773)" (#3289)
    • Fixing an issue that type 'null' and 'undefined' is not assignable to validateStatus when typescript strict option is enabled (#3200)

    Internal and Tests:

    • Lock travis to not use node v15 (#3361)

    Documentation:

    • Fixing simple typo, existant -> existent (#3252)
    • Fixing typos (#3309)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    ... (truncated)

    Changelog

    Sourced from axios's changelog.

    0.21.1 (December 21, 2020)

    Fixes and Functionality:

    • Hotfix: Prevent SSRF (#3410)
    • Protocol not parsed when setting proxy config from env vars (#3070)
    • Updating axios in types to be lower case (#2797)
    • Adding a type guard for AxiosError (#2949)

    Internal and Tests:

    • Remove the skipping of the socket http test (#3364)
    • Use different socket for Win32 test (#3375)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    0.21.0 (October 23, 2020)

    Fixes and Functionality:

    • Fixing requestHeaders.Authorization (#3287)
    • Fixing node types (#3237)
    • Fixing axios.delete ignores config.data (#3282)
    • Revert "Fixing overwrite Blob/File type as Content-Type in browser. (#1773)" (#3289)
    • Fixing an issue that type 'null' and 'undefined' is not assignable to validateStatus when typescript strict option is enabled (#3200)

    Internal and Tests:

    • Lock travis to not use node v15 (#3361)

    Documentation:

    • Fixing simple typo, existant -> existent (#3252)
    • Fixing typos (#3309)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • What does Eposide id correspond to in a recommendation dataset

    What does Eposide id correspond to in a recommendation dataset

    I have a recommender dataset where I have userID, user_feature1,user_featureX...., ItemID,Item_feature1...Item_featureY.., reward(sales amount),datetime.. how should I restructure this data to be used/tested in Minerva? In the tutorial, https://minerva-ui.readthedocs.io/en/v0.40/getting_started.html, I am unable to correlate what will be the eposide ID in my dataset.

    opened by ayush488 0
  • ERROR: (sqlite3.OperationalError) unable to open database file.

    ERROR: (sqlite3.OperationalError) unable to open database file.

    Hi,

    I just installed the Minerva. However when I execute minerva run command I receive following error:

    sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file (Background on this error at: http://sqlalche.me/e/14/e3q8)

    Can you please tell me what is causing this error and how can I fix it? Thanks in advance.

    bug 
    opened by Zaharah 3
Releases(v0.40)
  • v0.40(May 29, 2021)

    Support d3rlpy v0.90!

    MINERVA now supports d3rlpy v0.90 and there are the highlights for MINERVA.

    • more precise CQL and BEAR implementations
    • predict_value is fixed for unscaled actions
    • CRR algorithm is available
    Source code(tar.gz)
    Source code(zip)
  • v0.32(Mar 6, 2021)

    a way to upload images is changed

    Previously, the image files are uploaded via selecting the directory that contains all the files. However this is an easy way, the browser limits the number of files due to the efficiency issue. So, from this version, you can upload zipped file that contains the image files. This is the more efficient and fast way. image

    Source code(tar.gz)
    Source code(zip)
  • v0.31(Mar 6, 2021)

  • v0.30(Mar 5, 2021)

    Support d3rlpy v0.70!

    MINERVA now supports d3rlpy v0.70, which has a lot of progress from the v0.41 that is the previous dependency.

    These are some highlights related to MINERVA.

    • the continuous action is automatically normalized based on the dataset statistics
    • discrete action option has been removed since the action-space is automatically detected
    • many other internal enhancements
    Source code(tar.gz)
    Source code(zip)
  • v0.20(Dec 21, 2020)

    Support d3rlpy v0.41!

    MINERVA now supports d3rlpy v0.41, which has a lot of progress from the v0.23 that is the previous dependency.

    These are the some highlights related to MINERVA.

    • extremely fast mini-batch creation
    • extremely fast frame stacking for image observation
    • extremely fast N-step TD calculation
    • new metrics
    • etc

    Algorithm Selection

    MINERVA now provides many many algorithms for both discrete and continuous control datasets. You can choose an algorithm at the project creation dialog.

    discrete algorithms

    • DQN
    • Double DQN
    • AWR
    • CQL
    • BCQ
    • SAC

    continuous algorithms

    • DDPG
    • TD3
    • SAC
    • BCQ
    • BEAR
    • CQL
    • AWR
    • AWAC
    • PLAS

    Of course, there is the Q function option to incorporate arbitrary algorithms with the powerful distributional Q functions.

    Q functions

    • mean
    • Quantile Regression
    • Implicit Quantile Network
    • Fully parameterized Quantile Function
    Source code(tar.gz)
    Source code(zip)
  • v0.12(Sep 9, 2020)

  • v0.11(Sep 9, 2020)

    Dataset Dashboard

    • upload datasets
      • discrete/continuous action-space
      • vector/image observations

    Project Dashboard

    • train with CQL with detailed configurations.

    Export Policy

    • Support TorchScript and ONNX.
    Source code(tar.gz)
    Source code(zip)
Owner
Takuma Seno
Machine learning engineer at Sony AI / Ph.D CS student at Keio University.
Takuma Seno
Python library containing BART query generation and BERT-based Siamese models for neural retrieval.

Neural Retrieval Embedding-based Zero-shot Retrieval through Query Generation leverages query synthesis over large corpuses of unlabeled text (such as

Amazon Web Services - Labs 35 Apr 14, 2022
Multiple Object Extraction from Aerial Imagery with Convolutional Neural Networks

This is an implementation of Volodymyr Mnih's dissertation methods on his Massachusetts road & building dataset and my original methods that are publi

Shunta Saito 255 Sep 07, 2022
Chinese named entity recognization with BiLSTM using Keras

Chinese named entity recognization (Bilstm with Keras) Project Structure ./ ├── README.md ├── data │   ├── README.md │   ├── data 数据集 │   │   ├─

1 Dec 17, 2021
PyTorch implementations of the NeRF model described in "NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis"

PyTorch NeRF and pixelNeRF NeRF: Tiny NeRF: pixelNeRF: This repository contains minimal PyTorch implementations of the NeRF model described in "NeRF:

Michael A. Alcorn 178 Dec 20, 2022
Dynamic Graph Event Detection

DyGED Dynamic Graph Event Detection Get Started pip install -r requirements.txt TODO Paper link to arxiv, and how to cite. Twitter Weather dataset tra

Mert KoĹźan 3 May 09, 2022
The repo for reproducing Seed-driven Document Ranking for Systematic Reviews: A Reproducibility Study

ECIR Reproducibility Paper: Seed-driven Document Ranking for Systematic Reviews: A Reproducibility Study This code corresponds to the reproducibility

ielab 3 Mar 31, 2022
Action Segmentation Evaluation

Reference Action Segmentation Evaluation Code This repository contains the reference code for action segmentation evaluation. If you have a bug-fix/im

5 May 22, 2022
The source codes for TME-BNA: Temporal Motif-Preserving Network Embedding with Bicomponent Neighbor Aggregation.

TME The source codes for TME-BNA: Temporal Motif-Preserving Network Embedding with Bicomponent Neighbor Aggregation. Our implementation is based on TG

2 Feb 10, 2022
IJCAI2020 & IJCV 2020 :city_sunrise: Unsupervised Scene Adaptation with Memory Regularization in vivo

Seg_Uncertainty In this repo, we provide the code for the two papers, i.e., MRNet:Unsupervised Scene Adaptation with Memory Regularization in vivo, IJ

Zhedong Zheng 348 Jan 05, 2023
This repository contains FEDOT - an open-source framework for automated modeling and machine learning (AutoML)

package tests docs license stats support This repository contains FEDOT - an open-source framework for automated modeling and machine learning (AutoML

National Center for Cognitive Research of ITMO University 482 Dec 26, 2022
PyTorch Implementation of Realtime Multi-Person Pose Estimation project.

PyTorch Realtime Multi-Person Pose Estimation This is a pytorch version of Realtime_Multi-Person_Pose_Estimation, origin code is here Realtime_Multi-P

Dave Fang 157 Nov 12, 2022
Hysterese plugin with two temperature offset areas

craftbeerpi4 plugin OffsetHysterese Temperatur-Steuerungs-Plugin mit zwei tempereaturbereich abhängigen Offsets. Installation sudo pip3 install https:

HappyHibo 1 Dec 21, 2021
Semantic Edge Detection with Diverse Deep Supervision

Semantic Edge Detection with Diverse Deep Supervision This repository contains the code for our IJCV paper: "Semantic Edge Detection with Diverse Deep

Yun Liu 12 Dec 31, 2022
Learning to Adapt Structured Output Space for Semantic Segmentation, CVPR 2018 (spotlight)

Learning to Adapt Structured Output Space for Semantic Segmentation Pytorch implementation of our method for adapting semantic segmentation from the s

Yi-Hsuan Tsai 782 Dec 30, 2022
Cleaned up code for DSTC 10: SIMMC 2.0 track: subtask 2: multimodal coreference resolution

UNITER-Based Situated Coreference Resolution with Rich Multimodal Input: arXiv MMCoref_cleaned Code for the MMCoref task of the SIMMC 2.0 dataset. Pre

Yichen (William) Huang 2 Dec 05, 2022
This is the official PyTorch implementation for "Mesa: A Memory-saving Training Framework for Transformers".

A Memory-saving Training Framework for Transformers This is the official PyTorch implementation for Mesa: A Memory-saving Training Framework for Trans

Zhuang AI Group 105 Dec 06, 2022
Official implementation of Self-supervised Image-to-text and Text-to-image Synthesis

Self-supervised Image-to-text and Text-to-image Synthesis This is the official implementation of Self-supervised Image-to-text and Text-to-image Synth

6 Jul 31, 2022
Mix3D: Out-of-Context Data Augmentation for 3D Scenes (3DV 2021)

Mix3D: Out-of-Context Data Augmentation for 3D Scenes (3DV 2021) Alexey Nekrasov*, Jonas Schult*, Or Litany, Bastian Leibe, Francis Engelmann Mix3D is

Alexey Nekrasov 189 Dec 26, 2022
Attentive Implicit Representation Networks (AIR-Nets)

Attentive Implicit Representation Networks (AIR-Nets) Preprint | Supplementary | Accepted at the International Conference on 3D Vision (3DV) teaser.mo

29 Dec 07, 2022
Implementation of CVPR'21: RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction

RfD-Net [Project Page] [Paper] [Video] RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction Yinyu Nie, Ji Hou, Xiaoguang Han, Matthi

Yinyu Nie 162 Jan 06, 2023