Lipschitz-constrained Unsupervised Skill Discovery

This repository is the official implementation of

Seohong Park, Jongwook Choi*, Jaekyeom Kim*, Honglak Lee, Gunhee Kim (*: equal contribution). Lipschitz-constrained Unsupervised Skill Discovery. In ICLR, 2022. [paper] [project page] [slides]

The implementation is based on Unsupervised Skill Discovery with Bottleneck Option Learning and garage.

Visit our project page for more results including videos.

Requirements

Python 3.7.8
MuJoCo 1.5

Examples

Install requirements:

pip install -r requirements.txt
pip install -e .
pip install -e garaged

Ant with 2-D continuous skills:

python tests/main.py --run_group EXP --env ant --max_path_length 200 --dim_option 2 --common_lr 0.0001 --seed 0 --normalizer_type ant_preset --use_gpu 1 --traj_batch_size 20 --n_parallel 8 --n_epochs_per_eval 5000 --n_thread 1 --model_master_dim 1024 --record_metric_difference 0 --n_epochs_per_tb 100 --n_epochs_per_save 50000 --n_epochs_per_pt_save 5000 --n_epochs_per_pkl_update 1000 --eval_record_video 1 --n_epochs 200001 --spectral_normalization 1 --n_epochs_per_log 50 --discrete 0 --num_random_trajectories 200 --sac_discount 0.99 --alpha 0.01 --sac_lr_a -1 --lr_te 3e-05 --sac_scale_reward 0 --max_optimization_epochs 1 --trans_minibatch_size 2048 --trans_optimization_epochs 4 --eval_plot_axis -50 50 -50 50

Ant with 16 discrete skills:

python tests/main.py --run_group EXP --env ant --max_path_length 200 --dim_option 16 --common_lr 0.0001 --seed 0 --normalizer_type ant_preset --use_gpu 1 --traj_batch_size 20 --n_parallel 8 --n_epochs_per_eval 5000 --n_thread 1 --model_master_dim 1024 --record_metric_difference 0 --n_epochs_per_tb 100 --n_epochs_per_save 50000 --n_epochs_per_pt_save 5000 --n_epochs_per_pkl_update 1000 --eval_record_video 1 --n_epochs 200001 --spectral_normalization 1 --n_epochs_per_log 50 --discrete 1 --num_random_trajectories 200 --sac_discount 0.99 --alpha 0.003 --sac_lr_a -1 --lr_te 3e-05 --sac_scale_reward 0 --max_optimization_epochs 1 --trans_minibatch_size 2048 --trans_optimization_epochs 4 --eval_plot_axis -50 50 -50 50

Humanoid with 2-D continuous skills:

python tests/main.py --run_group EXP --env humanoid --max_path_length 1000 --dim_option 2 --common_lr 0.0003 --seed 0 --normalizer_type humanoid_preset --use_gpu 1 --traj_batch_size 5 --n_parallel 8 --n_epochs_per_eval 5000 --n_thread 1 --model_master_dim 1024 --record_metric_difference 0 --n_epochs_per_tb 100 --n_epochs_per_save 50000 --n_epochs_per_pt_save 5000 --n_epochs_per_pkl_update 1000 --eval_record_video 1 --n_epochs 200001 --spectral_normalization 1 --n_epochs_per_log 50 --discrete 0 --video_skip_frames 3 --num_random_trajectories 200 --sac_discount 0.99 --alpha 0.03 --sac_lr_a -1 --lr_te 0.0001 --lsd_alive_reward 0.03 --sac_scale_reward 0 --max_optimization_epochs 1 --trans_minibatch_size 2048 --trans_optimization_epochs 4 --sac_replay_buffer 1 --te_max_optimization_epochs 1 --te_trans_optimization_epochs 2

Humanoid with 16 discrete skills:

python tests/main.py --run_group EXP --env humanoid --max_path_length 1000 --dim_option 16 --common_lr 0.0003 --seed 0 --normalizer_type humanoid_preset --use_gpu 1 --traj_batch_size 5 --n_parallel 8 --n_epochs_per_eval 5000 --n_thread 1 --model_master_dim 1024 --record_metric_difference 0 --n_epochs_per_tb 100 --n_epochs_per_save 50000 --n_epochs_per_pt_save 5000 --n_epochs_per_pkl_update 1000 --eval_record_video 1 --n_epochs 200001 --spectral_normalization 1 --n_epochs_per_log 50 --discrete 1 --video_skip_frames 3 --num_random_trajectories 200 --sac_discount 0.99 --alpha 0.03 --sac_lr_a -1 --lr_te 0.0001 --lsd_alive_reward 0.03 --sac_scale_reward 0 --max_optimization_epochs 1 --trans_minibatch_size 2048 --trans_optimization_epochs 4 --sac_replay_buffer 1 --te_max_optimization_epochs 1 --te_trans_optimization_epochs 2

HalfCheetah with 8 discrete skills:

python tests/main.py --run_group EXP --env half_cheetah --max_path_length 200 --dim_option 8 --common_lr 0.0001 --seed 0 --normalizer_type half_cheetah_preset --use_gpu 1 --traj_batch_size 20 --n_parallel 8 --n_epochs_per_eval 5000 --n_thread 1 --model_master_dim 1024 --record_metric_difference 0 --n_epochs_per_tb 100 --n_epochs_per_save 50000 --n_epochs_per_pt_save 5000 --n_epochs_per_pkl_update 1000 --eval_record_video 1 --n_epochs 200001 --spectral_normalization 1 --n_epochs_per_log 50 --discrete 1 --num_random_trajectories 200 --sac_discount 0.99 --alpha 0.01 --sac_lr_a -1 --lr_te 3e-05 --sac_scale_reward 0 --max_optimization_epochs 1 --trans_minibatch_size 2048 --trans_optimization_epochs 4

Lipschitz-constrained Unsupervised Skill Discovery

Related tags

Overview

Lipschitz-constrained Unsupervised Skill Discovery

Requirements

Examples

Owner

Seohong Park

The AWS Certified SysOps Administrator

Like ThreeJS but for Python and based on wgpu

L-Verse: Bidirectional Generation Between Image and Text

NLU Dataset Diagnostics

[WACV 2022] Contextual Gradient Scaling for Few-Shot Learning

Code to reproduce the experiments from our NeurIPS 2021 paper " The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective"

Easy-to-use,Modular and Extendible package of deep-learning based CTR models .

Activating More Pixels in Image Super-Resolution Transformer

Python calculations for the position of the sun and moon.

Combining Reinforcement Learning and Constraint Programming for Combinatorial Optimization

Image-Stitching - Panorama composition using SIFT Features and a custom implementaion of RANSAC algorithm

Implementation of paper "Graph Condensation for Graph Neural Networks"

Pytorch implementation of "Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling"

The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks

🚀 An end-to-end ML applications using PyTorch, W&B, FastAPI, Docker, Streamlit and Heroku

A TensorFlow 2.x implementation of Masked Autoencoders Are Scalable Vision Learners

Deep Learning to Create StepMania SM FIles

🌎 The Modern Declarative Data Flow Framework for the AI Empowered Generation.

Official implementation for the paper: Multi-label Classification with Partial Annotations using Class-aware Selective Loss

A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.