Robot Action Primitives (RAPS)
This repository is the official implementation of Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives (RAPS).
[Project Website]
Murtaza Dalal, Deepak Pathak*, Ruslan Salakhutdinov*
(* equal advising)
CMU
If you find this work useful in your research, please cite:
@inproceedings{dalal2021raps,
Author = {Dalal, Murtaza and Pathak, Deepak and
Salakhutdinov, Ruslan},
Title = {Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives},
Booktitle = {NeurIPS},
Year = {2021}
}
Requirements
To install dependencies, please run the following commands:
sudo apt-get update
sudo apt-get install curl \
git \
libgl1-mesa-dev \
libgl1-mesa-glx \
libglew-dev \
libosmesa6-dev \
software-properties-common \
net-tools \
unzip \
vim \
virtualenv \
wget \
xpra \
xserver-xorg-dev
sudo apt-get install libglfw3-dev libgles2-mesa-dev patchelf
sudo mkdir /usr/lib/nvidia-000
Please add the following to your bashrc:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/.mujoco/mujoco200/bin
export MUJOCO_GL='egl'
export MKL_THREADING_LAYER=GNU
export D4RL_SUPPRESS_IMPORT_ERROR='1'
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/nvidia-000
To install python requirements:
conda create -n raps python=3.7
conda activate raps
./setup_python_env.sh <absolute path to raps>
Training and Evaluation
Kitchen
Prior to running any experiments, make sure to run cd /path/to/raps/rlkit
single task env names:
- microwave
- kettle
- slide_cabinet
- hinge_cabinet
- light_switch
- top_left_burner
multi task env names:
- microwave_kettle_light_top_left_burner //Sequential Multi Task 1
- hinge_slide_bottom_left_burner_light //Sequential Multi Task 2
To train RAPS with Dreamer on any single task kitchen environment, run:
python experiments/kitchen/dreamer/dreamer_v2_single_task_primitives.py --mode here_no_doodad --exp_prefix <> --env <env name>
To train RAPS with Dreamer on the multi task kitchen environments, run:
python experiments/kitchen/dreamer/dreamer_v2_multi_task_primitives.py --mode here_no_doodad --exp_prefix <> --env <env name>
To train Raw Actions with Dreamer on any kitchen environment
python experiments/kitchen/dreamer/dreamer_v2_raw_actions.py --mode here_no_doodad --exp_prefix <> --env <env name>
To train RAPS with RAD on any single task kitchen environment
python experiments/kitchen/rad/rad_single_task_primitives.py --mode here_no_doodad --exp_prefix <> --env <env name>
To train RAPS with RAD on any multi task kitchen environment
python experiments/kitchen/rad/rad_multi_task_primitives.py --mode here_no_doodad --exp_prefix <> --env <env name>
To train Raw Actions with RAD on any kitchen environment
python experiments/kitchen/rad/rad_raw_actions.py --mode here_no_doodad --exp_prefix <> --env <env name>
To train RAPS with PPO on any single task kitchen environment
python experiments/kitchen/ppo/ppo_single_task_primitives.py --mode here_no_doodad --exp_prefix <> --env <env name>
To train RAPS with PPO on any multi task kitchen environment
python experiments/kitchen/ppo/ppo_multi_task_primitives.py --mode here_no_doodad --exp_prefix <> --env <env name>
To train Raw Actions with PPO on any kitchen environment
python experiments/kitchen/ppo/ppo_raw_actions.py --mode here_no_doodad --exp_prefix <> --env <env name>
Metaworld
single task env names
- drawer-close-v2
- soccer-v2
- peg-unplug-side-v2
- sweep-into-v2
- assembly-v2
- disassemble-v2
To train RAPS with Dreamer on any metaworld environment
python experiments/metaworld/dreamer/dreamer_v2_single_task_primitives.py --mode here_no_doodad --exp_prefix <> --env <env name>
To train Raw Actions with Dreamer on any metaworld environment
python experiments/metaworld/dreamer/dreamer_v2_single_task_raw_actions.py --mode here_no_doodad --exp_prefix <> --env <env name>
Robosuite
To train RAPS with Dreamer on an Robosuite Lift
python experiments/robosuite/dreamer/dreamer_v2_single_task_primitives_lift.py --mode here_no_doodad --exp_prefix <>
To train Raw Actions with Dreamer on an Robosuite Lift
python experiments/robosuite/dreamer/dreamer_v2_single_task_raw_actions_lift.py --mode here_no_doodad --exp_prefix <>
To train RAPS with Dreamer on an Robosuite Door
python experiments/robosuite/dreamer/dreamer_v2_single_task_primitives_door.py --mode here_no_doodad --exp_prefix <>
To train Raw Actions with Dreamer on an Robosuite Door
python experiments/robosuite/dreamer/dreamer_v2_single_task_raw_actions_door.py --mode here_no_doodad --exp_prefix <>
Learning Curve visualization
cd /path/to/raps/rlkit
python ../viskit/viskit/frontend.py data/<exp_prefix> //open localhost:5000 to view
47 Dec 16, 2022
313 Dec 22, 2022
4 Jun 09, 2022
83 Dec 27, 2022
13 Dec 10, 2022
2 Sep 29, 2022
22 Aug 31, 2022
22 Sep 28, 2022
44 Nov 05, 2022
39 Dec 05, 2022
139 Dec 29, 2022
43 Sep 12, 2022
52 Nov 20, 2022
0 Jul 15, 2021
152 Dec 22, 2022
8 Aug 10, 2022
4 Sep 02, 2022
381 Jan 01, 2023
0 Feb 17, 2022
3.1k Jan 09, 2023