Anonymous implementation of KSL

Last update: Nov 10, 2021

Related tags

Deep Learning ksl

Overview

k-Step Latent (KSL)

Implementation of k-Step Latent (KSL) in PyTorch.

Representation Learning for Data-Efficient Reinforcement Learning

[Paper]

Code is built on top of the DrQ repo from Denis Yarats.

Getting Started

First, create and activate conda env:

conda env create -f conda_env.yml

conda activate ksl

This repo relies on environments from DMControl, and therefore assumes that you can run MuJoCo.

From within ./ksl, simply run:

python train.py

Altering training schemes can be done by feeding additional args, such as:

python train.py env=cheetah_run lr=2e-4

For a full list of customizable args, see ./ksl/configs.yaml.

Observing Runs

Just as in the DrQ repo, train.py will produce the runs folder, where all the outputs are going to be stored including train/eval logs, tensorboard blobs, and evaluation episode videos. To launch tensorboard run

tensorboard --logdir runs

The console output is also available in a form:

| train | E: 5 | S: 5000 | R: 11.4359 | D: 66.8 s | BR: 0.0581 | ALOSS: -1.0640 | CLOSS: 0.0996 | TLOSS: -23.1683 | TVAL: 0.0945 | AENT: 3.8132

a training entry decodes as

train - training episode
E - total number of episodes
S - total number of environment steps
R - episode return
D - duration in seconds
BR - average reward of a sampled batch
ALOSS - average loss of the actor
CLOSS - average loss of the critic
TLOSS - average loss of the temperature parameter
TVAL - the value of temperature
AENT - the actor's entropy

while an evaluation entry

| eval  | E: 20 | S: 20000 | R: 10.9356

contains

E - evaluation was performed after E episodes
S - evaluation was performed after S environment steps
R - average episode return computed over `num_eval_episodes` (usually 10)

Anonymous implementation of KSL

Related tags

Overview

k-Step Latent (KSL)

Getting Started

Observing Runs

Owner

A Temporal Extension Library for PyTorch Geometric

Some useful blender add-ons for SMPL skeleton's poses and global translation.

DziriBERT: a Pre-trained Language Model for the Algerian Dialect

The Official PyTorch Implementation of "LSGM: Score-based Generative Modeling in Latent Space" (NeurIPS 2021)

Mesh TensorFlow: Model Parallelism Made Easier

DeepLM: Large-scale Nonlinear Least Squares on Deep Learning Frameworks using Stochastic Domain Decomposition (CVPR 2021)

Repository containing the PhD Thesis "Formal Verification of Deep Reinforcement Learning Agents"

Implementation of Squeezenet in pytorch, pretrained models on Cifar 10 data to come

A framework for multi-step probabilistic time-series/demand forecasting models

A collection of resources and papers on Diffusion Models, a darkhorse in the field of Generative Models

Omniscient Video Super-Resolution

PyTorch reimplementation of minimal-hand (CVPR2020)

Implementation supporting the ICCV 2017 paper "GANs for Biological Image Synthesis"

Efficient Sharpness-aware Minimization for Improved Training of Neural Networks

Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning using 🤗 transformers

Sequence to Sequence Models with PyTorch

Monk is a low code Deep Learning tool and a unified wrapper for Computer Vision.

The source code of the ICCV2021 paper "PIRenderer: Controllable Portrait Image Generation via Semantic Neural Rendering"

Python scripts performing class agnostic object localization using the Object Localization Network model in ONNX.

A PyTorch Implementation of FaceBoxes