Deep Reinforcement Learning for Multiplayer Online Battle Arena

Last update: Dec 18, 2022

Related tags

Deep Learning MOBA_RL

Overview

MOBA_RL

Deep Reinforcement Learning for Multiplayer Online Battle Arena

Prerequisite

Python 3
gym-derk
Tensorflow 2.4.1
Dotaservice of TimZaman
Seed RL of Google
Ubuntu 20.04
RTX 3060 GPU, 16GB RAM is used to run Dota2 environment with rendering
RTX 3080 GPU, 46GB RAM is used to training 16 number of headless Dota2 environment together in my case

Derk Environment

We are going to train small MOBA environment called Derk.

First, move to dr-derks-mutant-battlegrounds folder.

Run below command to run the 50 parallel environemnt. I modified Seel_RL of Google for my MOBA case.

$ python learner_1.py --workspace_path [your path]/dr-derks-mutant-battlegrounds/
$ python learner_2.py --workspace_path [your path]/dr-derks-mutant-battlegrounds/
$ python run.py -p1 bot -p2 oldbot -n 50

You can check the training progress using Tensorboard log under tboard path of workspace.

Dota2 Environment

Rendering Environment

You first need to install Dota 2 from Steam. After installation, please check there is Dota2 folder under /home/[your account]/.steam/steam/steamapps/common/dota 2 beta'. We are going to run Dota2 from terminal command.

Next, you need to download and install dotaservice. In my case, I should modity the _run_dota function of dotaservice.py like below.

async def _run_dota(self):
  script_path = os.path.join(self.dota_path, self.DOTA_SCRIPT_FILENAME)
  script_path = '/home/kimbring2/.local/share/Steam/ubuntu12_32/steam-runtime/run.sh'

  # TODO(tzaman): all these options should be put in a proto and parsed with gRPC Config.
  args = [
       script_path,
       '/home/kimbring2/.local/share/Steam/steamapps/common/dota 2 beta/game/dota.sh',
       '-botworldstatesocket_threaded',
       '-botworldstatetosocket_frames', '{}'.format(self.ticks_per_observation),
       '-botworldstatetosocket_radiant', '{}'.format(self.PORT_WORLDSTATES[TEAM_RADIANT]),
       '-botworldstatetosocket_dire', '{}'.format(self.PORT_WORLDSTATES[TEAM_DIRE]),
       '-con_logfile', 'scripts/vscripts/bots/{}'.format(self.CONSOLE_LOG_FILENAME),
       '-con_timestamp',
       '-console',
       '-dev',
       '-insecure',
       '-noip',
       '-nowatchdog',  # WatchDog will quit the game if e.g. the lua api takes a few seconds.
       '+clientport', '27006',  # Relates to steam client.
       '+dota_1v1_skip_strategy', '1',
       '+dota_surrender_on_disconnect', '0',
       '+host_timescale', '{}'.format(self.host_timescale),
       '+hostname dotaservice',
       '+sv_cheats', '1',
       '+sv_hibernate_when_empty', '0',
       '+tv_delay', '0',
       '+tv_enable', '1',
       '+tv_title', '{}'.format(self.game_id),
       '+tv_autorecord', '1',
       '+tv_transmitall', '1',  # TODO(tzaman): what does this do exactly?
  ]

Training Environment

You need to build the Docker image of Dotaservice mentioned in README of Docker of the dotaservice.

You can run the Seel RL for Dota2 using below command.

$ ./run_dotaservice.sh 16
$ ./run_impala.sh 16

Addidinally, you can terminate all process using below command.

$ ./stop.sh

Deep Reinforcement Learning for Multiplayer Online Battle Arena

Related tags

Overview

MOBA_RL

Prerequisite

Derk Environment

Dota2 Environment

Rendering Environment

Training Environment

Owner

Dohyeong Kim

VSR-Transformer - This paper proposes a new Transformer for video super-resolution (called VSR-Transformer).

Implementation of the Triangle Multiplicative module, used in Alphafold2 as an efficient way to mix rows or columns of a 2d feature map, as a standalone package for Pytorch

PointPillars inference with TensorRT

Generative Autoregressive, Normalized Flows, VAEs, Score-based models (GANVAS)

Deep learning model, heat map, data prepo

Deep GPs built on top of TensorFlow/Keras and GPflow

AAI supports interdisciplinary research to help better understand human, animal, and artificial cognition.

exponential adaptive pooling for PyTorch

Most popular metrics used to evaluate object detection algorithms.

Sequential GCN for Active Learning

Tutorial page of the Climate Hack, the greatest hackathon ever

The implementation of the CVPR2021 paper "Structure-Aware Face Clustering on a Large-Scale Graph with 10^7 Nodes"

NCNN implementation of Real-ESRGAN. Real-ESRGAN aims at developing Practical Algorithms for General Image Restoration.

Pytorch implementation of the paper "Optimization as a Model for Few-Shot Learning"

Multimodal Co-Attention Transformer (MCAT) for Survival Prediction in Gigapixel Whole Slide Images

🏅 The Most Comprehensive List of Kaggle Solutions and Ideas 🏅

Code for Deterministic Neural Networks with Appropriate Inductive Biases Capture Epistemic and Aleatoric Uncertainty

Unified Pre-training for Self-Supervised Learning and Supervised Learning for ASR

Code for the paper "Query Embedding on Hyper-relational Knowledge Graphs"

Code for the Higgs Boson Machine Learning Challenge organised by CERN & EPFL