[NeurIPS 2021] "Delayed Propagation Transformer: A Universal Computation Engine towards Practical Control in Cyber-Physical Systems"

Related tags

Deep LearningDePT
Overview

Delayed Propagation Transformer: A Universal Computation Engine towards Practical Control in Cyber-Physical Systems

Introduction

Multi-agent control is a central theme in the Cyber-Physical Systems (CPS). However, current control methods either receive non-Markovian states due to insufficient sensing and decentralized design, or suffer from poor convergence. This paper presents the Delayed Propagation Transformer (DePT), a new transformer-based model that specializes in the global modeling of CPS while taking into account the immutable constraints from the physical world. DePT induces a cone-shaped spatial-temporal attention prior, which injects the information propagation and aggregation principles and enables a global view. With physical constraint inductive bias baked into its design, our DePT is ready to plug and play for a broad class of multi-agent systems. The experimental results on one of the most challenging CPS -- network-scale traffic signal control system in the open world -- demonstrated the superior performance of DePT on synthetic and real-world datasets.

Method

flow

scenario

tu

Installation Guide

The RL training loop of this repo is inherited from Colight repo: https://github.com/wingsweihua/colight

First, create new environment

This step is optional. CoLight (teacher model for DePT with imitation learning) requires tensorflow==1.x.

conda create -y -n 
   
     python=3.6
conda activate 
    

    
   

Then, install cityflow

Follow the [Official installation guide]

Or optionally, use the following commands without docker (docker is recommended but not mandatory)

git clone https://github.com/cityflow-project/CityFlow.git
cd CityFlow
pip install .

To test if you have successfully installed cityflow, check if the following python codes can pass without error:

import cityflow
eng = cityflow.Engine

Then, install requirements for teacher Colight

The RL training loop of DePT is based on Colight, they share the same dependencies. A complete environment that passed the test is provided in DePT/requirements.txt.

Training Guide

First, train teacher Colight:

set use_DePT = False in DePT/config.py, then run main.py

Second, pre-fit attention prior

Initialize model and pre-fit the priors using /DePT/DePT_src/pretrain_decayer.py

If downgrading DePT to transformer and not using the spatial tempooral cone shaped prior, skip this step.

Before training, keep track of the following configurations for training DePT:

If training a colight teacher model, set use_DePT = False in DePT/config.py: DIC_COLIGHT_AGENT_CONF. If training the DePT model, set it to False.

If enabling the spatial temporal cone shaped prior (default is enabled), set the following in DePT/model.py.

ablation1_cone = False
ablation2_time = False
only_1cone = False

If using Colight as the teacher model, set which_teacher='colight' in DePT/DePT_src/model.py, and set colight_fname to the pre-trained Colight teacher .h5 file.

Train DePT:

Example commands
python main.py 

python main.py --cnt 3600  --rounds 100  --gen 4  

python main.py --cnt 3600  --rounds 100  --gen 5  --volume='newyork' --road_net='28_7' --suffix='real_triple'

parameter meaning:

--rounds will specify the number of rounds generated, each round is 1 hour simulation time; 100 rounds are recommended.

--gen will specify number of generators; all generators work in parallel. 1 to 5 are recommended.

Simulation Platform that passed the test:

Ubuntu 20.04.2

RTX A6000

Driver Version: 460.91.03 CUDA Version: 11.2

Optional step before training:

Delete the following dirs (Automatically generated files) won't cause error in training, except losing your redundant training histories.

rm -rf model 
rm -rf records

Citation

comming soon.
Owner
VITA
Visual Informatics Group @ University of Texas at Austin
VITA
A Topic Modeling toolbox

Topik A Topic Modeling toolbox. Introduction The aim of topik is to provide a full suite and high-level interface for anyone interested in applying to

Anaconda, Inc. (formerly Continuum Analytics, Inc.) 93 Dec 01, 2022
Generating retro pixel game characters with Generative Adversarial Networks. Dataset "TinyHero" included.

pixel_character_generator Generating retro pixel game characters with Generative Adversarial Networks. Dataset "TinyHero" included. Dataset TinyHero D

Agnieszka Mikołajczyk 88 Nov 17, 2022
Auto-Encoding Score Distribution Regression for Action Quality Assessment

DAE-AQA It is an open source program reference to paper Auto-Encoding Score Distribution Regression for Action Quality Assessment. 1.Introduction DAE

13 Nov 16, 2022
Download and preprocess popular sequential recommendation datasets

Sequential Recommendation Datasets This repository collects some commonly used sequential recommendation datasets in recent research papers and provid

125 Dec 06, 2022
Implementation of Barlow Twins paper

barlowtwins PyTorch Implementation of Barlow Twins paper: Barlow Twins: Self-Supervised Learning via Redundancy Reduction This is currently a work in

IgorSusmelj 86 Dec 20, 2022
U^2-Net - Portrait matting This repository explores possibilities of using the original u^2-net model for portrait matting.

U^2-Net - Portrait matting This repository explores possibilities of using the original u^2-net model for portrait matting.

Dennis Bappert 104 Nov 25, 2022
GPU-accelerated Image Processing library using OpenCL

pyclesperanto pyclesperanto is a python package for clEsperanto - a multi-language framework for GPU-accelerated image processing. clEsperanto uses Op

17 Dec 25, 2022
This is the official repository of XVFI (eXtreme Video Frame Interpolation)

XVFI This is the official repository of XVFI (eXtreme Video Frame Interpolation), https://arxiv.org/abs/2103.16206 Last Update: 20210607 We provide th

Jihyong Oh 195 Dec 29, 2022
Tensorflow python implementation of "Learning High Fidelity Depths of Dressed Humans by Watching Social Media Dance Videos"

Learning High Fidelity Depths of Dressed Humans by Watching Social Media Dance Videos This repository is the official tensorflow python implementation

Yasamin Jafarian 287 Jan 06, 2023
This folder contains the python code of UR5E's advanced forward kinematics model.

This folder contains the python code of UR5E's advanced forward kinematics model. By entering the angle of the joint of UR5e, the detailed coordinates of up to 48 points around the robot arm can be c

Qiang Wang 4 Sep 17, 2022
Negative Interactions for Improved Collaborative Filtering:

Negative Interactions for Improved Collaborative Filtering: Don’t go Deeper, go Higher This notebook provides an implementation in Python 3 of the alg

Harald Steck 21 Mar 05, 2022
Facilitates implementing deep neural-network backbones, data augmentations

Introduction Nowadays, the training of Deep Learning models is fragmented and unified. When AI engineers face up with one specific task, the common wa

40 Dec 29, 2022
The official code of Anisotropic Stroke Control for Multiple Artists Style Transfer

ASMA-GAN Anisotropic Stroke Control for Multiple Artists Style Transfer Proceedings of the 28th ACM International Conference on Multimedia The officia

Six_God 146 Nov 21, 2022
[CVPR 2022] Back To Reality: Weak-supervised 3D Object Detection with Shape-guided Label Enhancement

Back To Reality: Weak-supervised 3D Object Detection with Shape-guided Label Enhancement Announcement 🔥 We have not tested the code yet. We will fini

Xiuwei Xu 7 Oct 30, 2022
TorchX is a library containing standard DSLs for authoring and running PyTorch related components for an E2E production ML pipeline.

TorchX is a library containing standard DSLs for authoring and running PyTorch related components for an E2E production ML pipeline

193 Dec 22, 2022
Supervised forecasting of sequential data in Python.

Supervised forecasting of sequential data in Python. Intro Supervised forecasting is the machine learning task of making predictions for sequential da

The Alan Turing Institute 54 Nov 15, 2022
We will see a basic program that is basically a hint to brute force attack to crack passwords. In other words, we will make a program to Crack Any Password Using Python. Show some ❤️ by starring this repository!

Crack Any Password Using Python We will see a basic program that is basically a hint to brute force attack to crack passwords. In other words, we will

Ananya Chatterjee 11 Dec 03, 2022
A collection of 100 Deep Learning images and visualizations

A collection of Deep Learning images and visualizations. The project has been developed by the AI Summer team and currently contains almost 100 images.

AI Summer 65 Sep 12, 2022
Probabilistic Gradient Boosting Machines

PGBM Probabilistic Gradient Boosting Machines (PGBM) is a probabilistic gradient boosting framework in Python based on PyTorch/Numba, developed by Air

Olivier Sprangers 112 Dec 28, 2022