Anonymous implementation of KSL

Last update: Nov 10, 2021

Related tags

Deep Learning ksl

Overview

k-Step Latent (KSL)

Implementation of k-Step Latent (KSL) in PyTorch.

Representation Learning for Data-Efficient Reinforcement Learning

[Paper]

Code is built on top of the DrQ repo from Denis Yarats.

Getting Started

First, create and activate conda env:

conda env create -f conda_env.yml

conda activate ksl

This repo relies on environments from DMControl, and therefore assumes that you can run MuJoCo.

From within ./ksl, simply run:

python train.py

Altering training schemes can be done by feeding additional args, such as:

python train.py env=cheetah_run lr=2e-4

For a full list of customizable args, see ./ksl/configs.yaml.

Observing Runs

Just as in the DrQ repo, train.py will produce the runs folder, where all the outputs are going to be stored including train/eval logs, tensorboard blobs, and evaluation episode videos. To launch tensorboard run

tensorboard --logdir runs

The console output is also available in a form:

| train | E: 5 | S: 5000 | R: 11.4359 | D: 66.8 s | BR: 0.0581 | ALOSS: -1.0640 | CLOSS: 0.0996 | TLOSS: -23.1683 | TVAL: 0.0945 | AENT: 3.8132

a training entry decodes as

train - training episode
E - total number of episodes
S - total number of environment steps
R - episode return
D - duration in seconds
BR - average reward of a sampled batch
ALOSS - average loss of the actor
CLOSS - average loss of the critic
TLOSS - average loss of the temperature parameter
TVAL - the value of temperature
AENT - the actor's entropy

while an evaluation entry

| eval  | E: 20 | S: 20000 | R: 10.9356

contains

E - evaluation was performed after E episodes
S - evaluation was performed after S environment steps
R - average episode return computed over `num_eval_episodes` (usually 10)

Anonymous implementation of KSL

Related tags

Overview

k-Step Latent (KSL)

Getting Started

Observing Runs

Owner

Python library for analysis of time series data including dimensionality reduction, clustering, and Markov model estimation

PyTorch common framework to accelerate network implementation, training and validation

Code for "Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistency" paper

Distributed DataLoader For Pytorch Based On Ray

Implementation for "Conditional entropy minimization principle for learning domain invariant representation features"

Supporting code for short YouTube series Neural Networks Demystified.

Spatial Transformer Nets in TensorFlow/ TensorLayer

Universal Adversarial Triggers for Attacking and Analyzing NLP (EMNLP 2019)

Azua - build AI algorithms to aid efficient decision-making with minimum data requirements.

NeurIPS'21 Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flows

Pynomial - a lightweight python library for implementing the many confidence intervals for the risk parameter of a binomial model

Send text to girlfriend in the morning

El-Gamal on Elliptic Curve (Python)

MinkLoc3D-SI: 3D LiDAR place recognition with sparse convolutions,spherical coordinates, and intensity

python debugger and anti-vm that checks if you're in a virtual machine or if someones trying to debug your file

NeRD: Neural Reflectance Decomposition from Image Collections

Anime Face Detector using mmdet and mmpose

Unofficial implementation of the Involution operation from CVPR 2021

This is an official source code for implementation on Extensive Deep Temporal Point Process

A library for implementing Decentralized Graph Neural Network algorithms.