Doom-based AI Research Platform for Reinforcement Learning from Raw Visual Information. :godmode:

Overview

ViZDoom PyPI version Build

ViZDoom allows developing AI bots that play Doom using only the visual information (the screen buffer). It is primarily intended for research in machine visual learning, and deep reinforcement learning, in particular.

ViZDoom is based on ZDoom to provide the game mechanics.

vizdoom_deadly_corridor

Features

  • Multi-platform (Linux, macOS, Windows),
  • API for Python, C++, Lua, Java and Julia (thanks to Jun Tian),
  • Easy-to-create custom scenarios (visual editors, scripting language and examples available),
  • Async and sync single-player and multi-player modes,
  • Fast (up to 7000 fps in sync mode, single threaded),
  • Lightweight (few MBs),
  • Customizable resolution and rendering parameters,
  • Access to the depth buffer (3D vision),
  • Automatic labeling game objects visible in the frame,
  • Off-screen rendering,
  • Episodes recording,
  • Time scaling in async mode.

ViZDoom API is reinforcement learning friendly (suitable also for learning from demonstration, apprenticeship learning or apprenticeship via inverse reinforcement learning, etc.).

Cite as

M Wydmuch, M Kempka & W Jaśkowski, ViZDoom Competitions: Playing Doom from Pixels, IEEE Transactions on Games, in print, arXiv:1809.03470

@article{wydmuch2018vizdoom,
  title={ViZDoom Competitions: Playing Doom from Pixels},
  author={Wydmuch, Marek and Kempka, Micha{\l} and Ja{\'s}kowski, Wojciech},
  journal={IEEE Transactions on Games},
  year={2018},
  publisher={IEEE}
}

or

M. Kempka, M. Wydmuch, G. Runc, J. Toczek & W. Jaśkowski, ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learning, IEEE Conference on Computational Intelligence and Games, pp. 341-348, Santorini, Greece, 2016 (arXiv:1605.02097)

@inproceedings{Kempka2016ViZDoom,
  author    = {Micha{\l} Kempka and Marek Wydmuch and Grzegorz Runc and Jakub Toczek and Wojciech Ja\'skowski},
  title     = {{ViZDoom}: A {D}oom-based {AI} Research Platform for Visual Reinforcement Learning},
  booktitle = {IEEE Conference on Computational Intelligence and Games},  
  year      = {2016},
  url       = {http://arxiv.org/abs/1605.02097},
  address   = {Santorini, Greece},
  Month     = {Sep},
  Pages     = {341--348},
  Publisher = {IEEE},
  Note      = {The best paper award}
}

Python quick start

Ubuntu

sudo apt install cmake libboost-all-dev libsdl2-dev libfreetype6-dev libgl1-mesa-dev libglu1-mesa-dev libpng-dev libjpeg-dev libbz2-dev libfluidsynth-dev libgme-dev libopenal-dev zlib1g-dev timidity tar nasm
pip install vizdoom

(we recommend using at least Ubuntu 18.04+ with Python 3.7+)

macOS

brew install cmake boost sdl2
pip install vizdoom

(we recommend using at least macOS High Sierra 10.13+ with Python 3.7+)

Windows build

For Windows we are providing compiled runtime binaries and development libraries:

1.1.8pre (2019-08-28):

See Installation of Windows binaries

Building instructions

Examples

Before running the provided examples, make sure that freedoom2.wad is placed in the same directory as the ViZDoom executable (on Linux and macOS it should be done automatically by the building process):

  • Python (contain learning examples implemented in PyTorch, TensorFlow and Theano)
  • C++
  • Julia

Python examples are currently the richest, so we recommend to look at them, even if you plan to use other language. API is almost identical for all languages.

See also the tutorial.

Documentation

Detailed description of all types and methods:

Additional documents:

Also full documentation of engine and ACS scripting language can be found on ZDoom Wiki.

Useful parts:

Awesome Doom tools

  • SLADE3 - great Doom map (scenario) editor for Linux, MacOS and Windows.
  • Doom Builder 2 - another great Doom map editor for Windows.
  • OBLIGE - Doom random map generator and PyOblige is a simple Python wrapper for it.
  • Omgifol - nice Python library for manipulating Doom maps.
  • NavDoom - Maze navigation generator for ViZDoom (similar to DeepMind Lab).
  • MazeExplorer - More sophisticated maze navigation generator for ViZDoom.
  • ViZDoomGym OpenAI Gym Wrapper for ViZDoom.

Contributions

This project is maintained and developed in our free time. All bug fixes, new examples, scenarios and other contributions are welcome! We are also open to features ideas and design suggestions.

License

Code original to ViZDoom is under MIT license. ZDoom uses code from several sources with varying licensing schemes.

Comments
  • Best way to organize self-play

    Best way to organize self-play

    I am planning to experiment with population-based training and self-play, similar to the recent DeepMind's Q3 CTF paper. The obvious requirement would be the ability to train the agents to play against other agent copies on the same map at the same time.

    I could probably wrap a multiplayer session into a single multi-agent interface and use ASYNC_PLAYER mode, maybe with increased tickrate (https://github.com/mwydmuch/ViZDoom/issues/209) However the optimal way to implement this would be to render multiple observations for different agents within the same tick in the same process in synchronous mode, similar to how it's done in single-player.

    Any thoughts on what is the right course of action here? Does multi-agent SYNC mode seem feasible or would it require changing half the codebase?

    question 
    opened by alex-petrenko 47
  • ERROR: Could not build wheels for vizdoom which use PEP 517 and cannot be installed directly

    ERROR: Could not build wheels for vizdoom which use PEP 517 and cannot be installed directly

    When I try to install the vizdoom using 'pip install',the error raised that : ERROR: Command errored out with exit status 1: command: /usr/bin/python3 /tmp/tmpoln7vkbf build_wheel /tmp/tmp6vsdacbc cwd: /tmp/pip-install-kby3x8wx/vizdoom Complete output (38 lines): running bdist_wheel running build Traceback (most recent call last): File "/tmp/tmpoln7vkbf", line 280, in main() File "/tmp/tmpoln7vkbf", line 263, in main json_out['return_val'] = hook(**hook_input['kwargs']) File "/tmp/tmpoln7vkbf", line 204, in build_wheel return _build_backend().build_wheel(wheel_directory, config_settings, File "/tmp/pip-build-env-jljcnwxe/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 230, in build_wheel return self._build_with_temp_dir(['bdist_wheel'], '.whl', File "/tmp/pip-build-env-jljcnwxe/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 215, in _build_with_temp_dir self.run_setup() File "/tmp/pip-build-env-jljcnwxe/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 267, in run_setup super(_BuildMetaLegacyBackend, File "/tmp/pip-build-env-jljcnwxe/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 158, in run_setup exec(compile(code, file, 'exec'), locals()) File "setup.py", line 147, in setup( File "/tmp/pip-build-env-jljcnwxe/overlay/lib/python3.8/site-packages/setuptools/init.py", line 153, in setup return distutils.core.setup(**attrs) File "/usr/lib/python3.8/distutils/core.py", line 148, in setup dist.run_commands() File "/usr/lib/python3.8/distutils/dist.py", line 966, in run_commands self.run_command(cmd) File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/tmp/pip-build-env-jljcnwxe/overlay/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 299, in run self.run_command('build') File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command cmd_obj.run() File "setup.py", line 125, in run if os.path.exists(python_library): File "/usr/lib/python3.8/genericpath.py", line 19, in exists os.stat(path) TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType ERROR: Failed building wheel for vizdoom Failed to build vizdoom ERROR: Could not build wheels for vizdoom which use PEP 517 and cannot be installed directly And my OS info lies below:

    Linux version 4.4.0-19041-Microsoft ([email protected]) (gcc version 5.4.0 (GCC) ) #1237-Microsoft Sat Sep 11 14:32:00 PST 2021

    bug building wsl 
    opened by Throne-myworld 41
  • Can't compile on macOS with brew-installed boost and boost-python

    Can't compile on macOS with brew-installed boost and boost-python

    I've just updated my ViZDoom version and even though it used to compile and work just fine, fbda2ab19 seems to break the build for me . With this change it can no longer find boost-python. I installed the libraries using the following commands. I see that /usr/local/include/boost/python.hpp exists.

    brew install boost
    brew install boost-python --with-python3
    

    Once I comment out this addtional set it compiles without any problems.

    bug macos/osx cmake 
    opened by apaszke 29
  • Added audio buffer

    Added audio buffer

    closes #225

    This work adds sound into the state space of vizdoom, we can now pass audio buffers synchronously as a NumPy array to the state variable. To enable this set game.set_soft_sound_enabled(True). enabling this slows down sync frame rate by a bit, but this gives researchers the option to tap into a lesser-seen state space.

    enhancement 
    opened by hegde95 25
  • Build error on Ubuntu 16.04 LTS amd64

    Build error on Ubuntu 16.04 LTS amd64

    I'm getting a link error at build on Ubuntu 16.04 LTS. Any ideas?

    [ 99%] Building CXX object src/vizdoom/src/CMakeFiles/vizdoom.dir/vizdoom_game.cpp.o
    [ 99%] Building CXX object src/vizdoom/src/CMakeFiles/vizdoom.dir/vizdoom_screen.cpp.o
    [ 99%] Building CXX object src/vizdoom/src/CMakeFiles/vizdoom.dir/vizdoom_shared_memory.cpp.o
    [100%] Building CXX object src/vizdoom/src/CMakeFiles/vizdoom.dir/vizdoom_message_queue.cpp.o
    [100%] Building CXX object src/vizdoom/src/CMakeFiles/vizdoom.dir/vizdoom_main.cpp.o
    [100%] Building CXX object src/vizdoom/src/CMakeFiles/vizdoom.dir/vizdoom_depth.cpp.o
    [100%] Linking CXX executable ../../../bin/vizdoom
    CMakeFiles/vizdoom.dir/tempfiles.cpp.o: In function `FTempFileName::FTempFileName(char const*)':
    tempfiles.cpp:(.text+0x7): warning: the use of `tempnam' is dangerous, better use `mkstemp'
    //lib/x86_64-linux-gnu/libsystemd.so.0: undefined reference to `[email protected]_5.0'
    //lib/x86_64-linux-gnu/libsystemd.so.0: undefined reference to `[email protected]_5.0'
    //lib/x86_64-linux-gnu/libsystemd.so.0: undefined reference to `[email protected]_5.0'
    collect2: error: ld returned 1 exit status
    src/vizdoom/src/CMakeFiles/vizdoom.dir/build.make:8868: recipe for target 'bin/vizdoom' failed
    make[2]: *** [bin/vizdoom] Error 1
    CMakeFiles/Makefile2:897: recipe for target 'src/vizdoom/src/CMakeFiles/vizdoom.dir/all' failed
    make[1]: *** [src/vizdoom/src/CMakeFiles/vizdoom.dir/all] Error 2
    Makefile:83: recipe for target 'all' failed
    make: *** [all] Error 2
    
    opened by hholst80 22
  • Randomness in ACS scripts

    Randomness in ACS scripts

    I've noticed that the function 'random' in acs scripts works poorly, since the numbers obtained have kind of a scheme and generally one number repeats many times more than the others.

    Do you guys know if there's a way to use a 'better' random number generator?

    Coming from ZDOOM I don't see any hope. No C++ functions can be imported or so. I could imagine however, that somehow one can set a variable in the acs script, directly from vizdoom. Something like, game.set_game_variable(USER1). Would it be possible? Do you see any workaround?

    Thanks in advance!

    enhancement design suggestion windows 
    opened by imrodriguezro 21
  • Suggestion: API/commands for fetching audio

    Suggestion: API/commands for fetching audio

    E.g. by commanding "enable_audio" before initing the game, and then receiving additional object in State object which holds audio samples played inside that time frame.

    I know it is VizDoom but this could possibly allow bots to "home in" towards high-action areas and/or hear close-by enemies behind them.

    enhancement help wanted feature request 
    opened by Miffyli 19
  • [Question] User Variables

    [Question] User Variables

    Hello,

    I can't find any documentation on the user variables like the one used in the example on your website:

    "It is also possible to access user variables (USER1, USER2 ... USER32), which correspond to ACS scripts' global variables 1-32." Such as when used in game.get_game_variable(GameVariable.USER1)

    I've tried looking at ACS documentation on ZDoom and Doom wikis, but I cannot find and reference to these global variables. Is there a list somewhere of these variables?

    I am asking because I want to make the agent lose reward whenever it uses ammo so it learns to conserve it.

    question 
    opened by zappybiby 19
  • Vizdoom navigation problem

    Vizdoom navigation problem

    Hi, I'm using VizDoom to train my agent to learn to navigate. But there are several questions:

    1. in my scenario, the maze is discrete, which means I regard the vizdoom as a 3D grid world. However, I don't know how to randomly initialize my agent in those grids.
    2. By the way, I want the agent to move in three ways: move forward for 1 grid (a specific length), turn left (90 degrees), and turn right (90 degrees as well). But I only find the button to move in a very small step or turn a small angle. Could you please provide me with some hints to do those things? Thanks a lot!
    question 
    opened by pengzhi1998 18
  • Adding wrapper for gym

    Adding wrapper for gym

    Adding Gym Wrapper for Vizdoom

    1. Base env, env definitions, documentation borrowed from vizdoomgym with minor changes wrt recent gym API (0.22.0).
    2. env.render is done using pygame after extracting it from vizdoom's screen buffer. Can maybe use Vizdoom API directly if gym rendering API is changed.
    3. Added a test to check the env for different obs types for every configuration. (health, position, labels, depth = True / False). Another test to check obs at terminal state, since this is handled separately.
    4. Unsure about the folder placement. I felt this was okay. Added commands to copy it to build destination.
    opened by arjun-kg 16
  • windows install(really rookie)

    windows install(really rookie)

    hi im try to use vizdoom on my computer os:windows10 i've install vizdoom in D:\Users\user\Anaconda3\Lib\site-packages\vizdoom and my basic.py is in D:\Users\user\Anaconda3\Lib\site-packages\vizdoom\examples use vscode to run this

    from vizdoom import * import random, time

    game = DoomGame() game.load_config("../config/basic.cfg") game.init()

    shoot = [0,0,1] left = [1,0,0] right = [0,1,0]

    episodes = 10 for i in range(episodes): game.new_episode() while not game.is_episode_finished(): state = game.get_state() img = state.image_buffer misc = state.game_variables action = random.choice([shoot, left, right]) # A random agent reward = game.make_action(action) time.sleep(0.02) # Just to make it slower print("Total reward: " + str(game.get_total_reward()))

    and got this D:#Git\Python>python vizdoom.py Traceback (most recent call last): File "vizdoom.py", line 1, in from vizdoom import * File "D:#Git\Python\vizdoom.py", line 4, in game = DoomGame() NameError: name 'DoomGame' is not defined

    im not sure what kind of problem am i running to

    since i dont really understand the installing step so any problem is possible to make by me, im very sorry for such unclear question,and ive been searching for solution for a long time and i still cant figure it out. if i miss any information for my question,please let me know. thank you for your apply.

    oh ! i can run vizdoom.exe,dont know would that help?

    opened by KnightOf 16
  • Fix SetuptoolsDeprecationWarning in setup.py

    Fix SetuptoolsDeprecationWarning in setup.py

    New versions of setuptools warns about:

    SetuptoolsDeprecationWarning:     Installing 'vizdoom.gym_wrapper' as data is deprecated, please list it in `packages`.
        !!
    
    
        ############################
        # Package would be ignored #
        ############################
        Python recognizes 'vizdoom.gym_wrapper' as an importable package,
        but it is not listed in the `packages` configuration of setuptools.
    
        'vizdoom.gym_wrapper' has been automatically added to the distribution only
        because it may contain data files, but this behavior is likely to change
        in future versions of setuptools (and therefore is considered deprecated).
    
        Please make sure that 'vizdoom.gym_wrapper' is included as a package by using
        the `packages` configuration field or the proper discovery methods
        (for example by using `find_namespace_packages(...)`/`find_namespace:`
        instead of `find_packages(...)`/`find:`).
    
        You can read more about "package discovery" and "data files" on setuptools
        documentation page.
    
    
    !!
    
      check.warn(importable)
    E:\Python37\lib\site-packages\setuptools\command\build_py.py:202: SetuptoolsDeprecationWarning:     Installing 'vizdoom.scenarios' as data is deprecated, please list it in `packages`.
        !!
    
    
        ############################
        # Package would be ignored #
        ############################
        Python recognizes 'vizdoom.scenarios' as an importable package,
        but it is not listed in the `packages` configuration of setuptools.
    
        'vizdoom.scenarios' has been automatically added to the distribution only
        because it may contain data files, but this behavior is likely to change
        in future versions of setuptools (and therefore is considered deprecated).
    
        Please make sure that 'vizdoom.scenarios' is included as a package by using
        the `packages` configuration field or the proper discovery methods
        (for example by using `find_namespace_packages(...)`/`find_namespace:`
        instead of `find_packages(...)`/`find:`).
    
        You can read more about "package discovery" and "data files" on setuptools
        documentation page.
    
    
    !!
    
    python farama standard 
    opened by mwydmuch 0
  • Add docstrings to pybind11 module

    Add docstrings to pybind11 module

    As in the title, this might be a nice addition. Not sure if it's possible to make it also work with Sphinx. They should use the pydocstyle and Google styles so that they have a standard formatting.

    farama standard 
    opened by mwydmuch 0
  • Create documentation in farama.org style

    Create documentation in farama.org style

    This consists of a few subtasks:

    • Create Farama-like documentation generated by sphinx.
    • Figure out what is a proper way to document Gym/Gymnasium API and core API (pybind11 module) and generate sphinx docs for them (and migrate old docs to the new format).
    • Have the documentation website be at .farama.org.
    • Include the standard Farama code of conduct.
    enhancement farama standard documentation 
    opened by mwydmuch 0
  • Deploy packages to PyPI via GitHub Actions

    Deploy packages to PyPI via GitHub Actions

    A workflow for building for Linux (manylinux?), macOS (Intel only?) and Windows (optional?) wheels using GitHub Actions (currently, ViZDoom is only distributed as source distribution, except for Windows, where wheels are provided) needs to be created. It should run when a Release is published on GitHub.

    enhancement python pip/conda building farama standard ci/cd 
    opened by mwydmuch 0
Releases(1.1.14)
Owner
Marek Wydmuch
Ph.D. student, machine learning and 3D graphics enthusiast
Marek Wydmuch
TF-Agents: A reliable, scalable and easy to use TensorFlow library for Contextual Bandits and Reinforcement Learning.

TF-Agents: A reliable, scalable and easy to use TensorFlow library for Contextual Bandits and Reinforcement Learning. TF-Agents makes implementing, de

2.4k Dec 29, 2022
This is the official implementation of Multi-Agent PPO.

MAPPO Chao Yu*, Akash Velu*, Eugene Vinitsky, Yu Wang, Alexandre Bayen, and Yi Wu. Website: https://sites.google.com/view/mappo This repository implem

653 Jan 06, 2023
Retro Games in Gym

Status: Maintenance (expect bug fixes and minor updates) Gym Retro Gym Retro lets you turn classic video games into Gym environments for reinforcement

OpenAI 2.8k Jan 03, 2023
ChainerRL is a deep reinforcement learning library built on top of Chainer.

ChainerRL ChainerRL is a deep reinforcement learning library that implements various state-of-the-art deep reinforcement algorithms in Python using Ch

Chainer 1.1k Dec 26, 2022
Rethinking the Importance of Implementation Tricks in Multi-Agent Reinforcement Learning

MARL Tricks Our codes for RIIT: Rethinking the Importance of Implementation Tricks in Multi-AgentReinforcement Learning. We implemented and standardiz

404 Dec 25, 2022
Tensorforce: a TensorFlow library for applied reinforcement learning

Tensorforce: a TensorFlow library for applied reinforcement learning Introduction Tensorforce is an open-source deep reinforcement learning framework,

Tensorforce 3.2k Jan 02, 2023
A fork of OpenAI Baselines, implementations of reinforcement learning algorithms

Stable Baselines Stable Baselines is a set of improved implementations of reinforcement learning algorithms based on OpenAI Baselines. You can read a

Ashley Hill 3.7k Jan 01, 2023
A customisable 3D platform for agent-based AI research

DeepMind Lab is a 3D learning environment based on id Software's Quake III Arena via ioquake3 and other open source software. DeepMind Lab provides a

DeepMind 6.8k Jan 05, 2023
A toolkit for developing and comparing reinforcement learning algorithms.

Status: Maintenance (expect bug fixes and minor updates) OpenAI Gym OpenAI Gym is a toolkit for developing and comparing reinforcement learning algori

OpenAI 29.6k Jan 01, 2023
Monitor your el-cheapo UPS via SNMP

UPSC-SNMP-Agent UPSC-SNMP-Agent exposes your el-cheapo locally connected UPS via the SNMP network management protocol. This enables various equipment

Tom Szilagyi 32 Jul 28, 2022
Open world survival environment for reinforcement learning

Crafter Open world survival environment for reinforcement learning. Highlights Crafter is a procedurally generated 2D world, where the agent finds foo

Danijar Hafner 213 Jan 05, 2023
Dopamine is a research framework for fast prototyping of reinforcement learning algorithms.

Dopamine Dopamine is a research framework for fast prototyping of reinforcement learning algorithms. It aims to fill the need for a small, easily grok

Google 10k Jan 07, 2023
A toolkit for reproducible reinforcement learning research.

garage garage is a toolkit for developing and evaluating reinforcement learning algorithms, and an accompanying library of state-of-the-art implementa

Reinforcement Learning Working Group 1.6k Jan 09, 2023
TensorFlow Reinforcement Learning

TRFL TRFL (pronounced "truffle") is a library built on top of TensorFlow that exposes several useful building blocks for implementing Reinforcement Le

DeepMind 3.1k Dec 29, 2022
Modular Deep Reinforcement Learning framework in PyTorch. Companion library of the book "Foundations of Deep Reinforcement Learning".

SLM Lab Modular Deep Reinforcement Learning framework in PyTorch. Documentation: https://slm-lab.gitbook.io/slm-lab/ BeamRider Breakout KungFuMaster M

Wah Loon Keng 1.1k Dec 24, 2022
A platform for Reasoning systems (Reinforcement Learning, Contextual Bandits, etc.)

Applied Reinforcement Learning @ Facebook Overview ReAgent is an open source end-to-end platform for applied reinforcement learning (RL) developed and

Facebook Research 3.3k Jan 05, 2023
Game Agent Framework. Helping you create AIs / Bots that learn to play any game you own!

Serpent.AI - Game Agent Framework (Python) Update: Revival (May 2020) Development work has resumed on the framework with the aim of bringing it into 2

Serpent.AI 6.4k Jan 05, 2023
OpenAI Baselines: high-quality implementations of reinforcement learning algorithms

Status: Maintenance (expect bug fixes and minor updates) Baselines OpenAI Baselines is a set of high-quality implementations of reinforcement learning

OpenAI 13.5k Jan 07, 2023
An open source robotics benchmark for meta- and multi-task reinforcement learning

Meta-World Meta-World is an open-source simulated benchmark for meta-reinforcement learning and multi-task learning consisting of 50 distinct robotic

Reinforcement Learning Working Group 823 Jan 06, 2023