The CARL Benchmark Library
CARL (context adaptive RL) provides highly configurable contextual extensions to several well-known RL environments. It's designed to test your agent's generalization capabilities in all scenarios where intra-task generalization is important.
Benchmarks include:
-
OpenAI gym classic control suite extended with several physics context features like gravity or friction
-
OpenAI gym Box2D BipedalWalker, LunarLander and CarRacing, each with their own modification possibilities like new vehicles to race
-
All Brax locomotion environments with exposed internal features like joint strength or torso mass
-
Super Mario (TOAD-GAN), a procedurally generated jump'n'run game with control over level similarity
-
RNADesign, an environment for RNA design given structure constraints with structures from different datasets to choose from
Installation
We recommend you use a virtual environment (e.g. Anaconda) to install CARL and its dependencies. We recommend and test with python 3.9 under Linux.
First, clone our repository and install the basic requirements:
git clone https://github.com/automl/CARL.git --recursive
cd CARL
pip install .
This will only install the basic classic control environments, which should run on most operating systems. For the full set of environments, use the install options:
pip install -e .[box2d, brax, rna, mario]
These may not be compatible with Windows systems. Box2D environment may need to be installed via conda on MacOS systems:
conda install -c conda-forge gym-box2d
In general, we test on Linux systems, but aim to keep the benchmark compatible with MacOS as much as possible. Mario at this point, however, will not run on any operation system besides Linux
To install the additional requirements for ToadGAN:
javac src/envs/mario/Mario-AI-Framework/**/*.java
If you want to use the RNA design environment:
cd src/envs/rna/learna
make requirements
make data
In case you want to run our experiments or use our training files, also install the experiment dependencies:
pip install -e .[experiments]
Train an Agent
To get started with CARL, you can use our 'train.py' script. It will train a PPO agent on the environment of your choice with custom context variations that are sampled from a standard deviation.
To use MetaCartPole with variations in gravity and friction by 20% compared to the default, run:
python train.py
--env CARLCartPoleEnv
--context_args gravity friction
--default_sample_std_percentage 0.2
--outdir <result_location>
You can use the plotting scripts in src/eval to view the results.
CARL's Contextual Extension
CARL contextually extends the environment by making the context visible and configurable. During training we therefore can encounter different contexts and train for generalization. We exemplarily show how Brax' Fetch is extended and embedded by CARL. Different instiations can be achieved by setting the context features to different values.
Cite Us
@misc{CARL,
author = {C. Benjamins and
T. Eimer and
F. Schubert and
A. Biedenkapp and
B. Rosenhahn and
F. Hutter and
M. Lindauer},
title = {CARL: A Benchmark for Contextual and Adaptive Reinforcement Learning},
howpublished = {https://github.com/automl/CARL},
year = {2021},
month = aug,
}
References
OpenAI gym, Brockman et al., 2016. arXiv preprint arXiv:1606.01540
TOAD-GAN: Coherent Style Level Generation from a Single Example, Awiszus et al., AIIDE 2020
Learning to Design RNA, Runge et al., ICRL 2019
License
CARL falls under the Apache License 2.0 (see file 'LICENSE') as is permitted by all work that we use. This includes CARLMario, which is not based on the Nintendo Game, but on TOAD-GAN and TOAD-GUI running under an MIT license. They in turn make use of the Mario AI framework (https://github.com/amidos2006/Mario-AI-Framework). This is not the original game but a replica, explicitly built for research purposes and includes a copyright notice (https://github.com/amidos2006/Mario-AI-Framework#copyrights ).
1.2k Dec 29, 2022
5 Dec 13, 2021
873 Jan 06, 2023
252 Sep 16, 2022
21 Dec 22, 2022
4 Sep 17, 2022
3 Sep 23, 2022
41 Nov 23, 2022
105 May 15, 2022
21 Jul 30, 2022
10 Nov 02, 2022
22 Dec 12, 2022
34 Nov 09, 2022
1 Jan 12, 2022
1 Dec 08, 2021
31 Nov 06, 2022
353 Dec 27, 2022
0 Feb 06, 2022
3 Apr 08, 2022
992 Dec 30, 2022