Code for our paper "MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction" published at ICCV 2021.

Last update: Jan 05, 2023

Related tags

Deep Learning MG-GAN

Overview

MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction

This repository contains the code for the paper

MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction
Patrick Dendorfer*, Sven Elflein*, Laura Leal-Taixé (* equal contribution)
International Conference on Computer Vision (ICCV), 2021

Motivation

The distribution over future trajectories of pedestrians is often multi-modal and does not have connected support (a).

We found that single generator GANs introduce out-of-distribution (OOD) samples in this case due to GANs mapping the continuous latent variable z with a continuous function (b). These OOD samples might introduce unforseen behavior in real world applications, such as autonomous driving.

To resolve this problem, we propose to learn the target distribution in a piecewise manner using multiple generators, effectively preventing OOD samples (c).

Model

Our model consists of four key components: Encoding modules, Attention modules, and our novel contribution PM-Network learning a distribution over multiple Generators.

Setup

First, setup Python environment

conda create -f environment.yml -n mggan
conda activate mggan

Then, download the datasets (data.zip) from here and unzip in the root of this repository

unzip data.zip

which will create a folder ./data/datasets.

Training

Models can be trained using the script mggan/model/train.py using the following command

python mggan/models/pinet_multi_generator/train.py --name <name_of_experiment> --num_gens <number_of_generators>  --dataset <dataset_name> --epochs 50

This generates a output folder in ./logs/<name_of_experiment> with Tensorboard logs and the model checkpoints. You can use tensorboard --logdir ./logs/<name_of_experiment> to monitor the training process.

Evaluation

For evaluation of metrics (ADE, FDE, Precison, Recall) for k=1 to k=20 predictions, use

python scripts/evaluate.py --model_path <path_to_model_directory>  --output_folder <folder_to_store_result_csv>

One can use --eval-set <dataset_name> to evaluate models on other test sets than the dataset the model was trained on. This is useful to evaluate the BIWI models on the Garden of Forking Paths dataset (gofp) for which we report results in the paper.

Pre-trained models

We provide pre-trained models for MG-GAN with 2-8 generators together with the training configurations, on the BIWI datasets and Stanford Drone dataset (SDD) here.

Citation

If our work is useful to you, please consider citing

@inproceedings{dendorfer2021iccv,
  title={MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction}, 
  author={Dendorfer, Patrick and Elflein, Sven and Leal-Taixé, Laura},
  month={October}
  year={2021},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  }

The implementation of the algorithm in the paper "Safe Deep Semi-Supervised Learning for Unseen-Class Unlabeled Data" published in ICML 2020.

DS3L This is the code for paper "Safe Deep Semi-Supervised Learning for Unseen-Class Unlabeled Data" published in ICML 2020. Setups The code is implem

36 Oct 19, 2022

This is the repo for the paper `SumGNN: Multi-typed Drug Interaction Prediction via Efficient Knowledge Graph Summarization'. (published in Bioinformatics'21)

SumGNN: Multi-typed Drug Interaction Prediction via Efficient Knowledge Graph Summarization This is the code for our paper ``SumGNN: Multi-typed Drug

58 Dec 21, 2022

Code for ICCV 2021 paper "Distilling Holistic Knowledge with Graph Neural Networks"

HKD Code for ICCV 2021 paper "Distilling Holistic Knowledge with Graph Neural Networks" cifia-100 result The implementation of compared methods are ba

30 Dec 18, 2022

code for ICCV 2021 paper 'Generalized Source-free Domain Adaptation'

G-SFDA Code (based on pytorch 1.3) for our ICCV 2021 paper 'Generalized Source-free Domain Adaptation'. [project] [paper]. Dataset preparing Download

84 Dec 26, 2022

Code for ICCV 2021 paper: ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators..

ARAPReg Code for ICCV 2021 paper: ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators.. Installation The cod

132 Nov 28, 2022

Comments

request to visualizer

Hello author！ I admire your work and would like to reproduce your results. There is a small requirement here that needs to trouble you. Do you have a visual code, which has shown the effect in your paper. Thanks again for your work and contributions！

opened by 12num 0
Question regarding Garden of Forking Path Dataset

Hello,

I see there are more scenes in the test set (ETH, Hotel, and ZARA1) than the train set (ETH) in your pre-processed dataset of GOFP. Could you kindly elaborate on why it is that?

Thanks, Sourav Das

opened by SodaCoder 0
Question about ETH&UCY Dataset

Hi, I notice that trajectories in some datasets are not consistent with provided in Social GAN. May I ask how do you preprocess your data? It will be helpful to conduct my experiments in a fair environment. Thanks!

opened by HRHLALALA 1
Reproducible MG-GAN code for the FPD dataset

Hello Patrick, Sven,

This is Sourav Das, a 1st year Ph.D. student at the University of Padova, Italy.

This Github repository has the reproducible implementation for the datasets: ETH, Hotel, Social_Stanford_Synthetic, Stanford, Univ, Zara1, Zara2, and GOFP.

I would like to reproduce the results on FPD datasets also. Could you kindly share with me the code with support for the FPD dataset?

Here is my Github: https://github.com/SodaCoder

Thanks in advance,

opened by SodaCoder 1

Releases(1.0)

1.0(Aug 14, 2021)

data.zip contains the datasets, checkpoints.zip contains pre-trained models for BIWI and Stanford Drone datasets.
Source code(tar.gz)
Source code(zip)
checkpoints.zip(296.92 MB)
data.zip(82.24 MB)

Code for our paper "MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction" published at ICCV 2021.

Related tags

Overview

MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction

Motivation

Model

Setup

Training

Evaluation

Pre-trained models

Citation

You might also like...

The implementation of the algorithm in the paper "Safe Deep Semi-Supervised Learning for Unseen-Class Unlabeled Data" published in ICML 2020.

This is the repo for the paper `SumGNN: Multi-typed Drug Interaction Prediction via Efficient Knowledge Graph Summarization'. (published in Bioinformatics'21)

Code for ICCV 2021 paper "Distilling Holistic Knowledge with Graph Neural Networks"

code for ICCV 2021 paper 'Generalized Source-free Domain Adaptation'

Code for ICCV 2021 paper: ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators..

Code for the ICCV 2021 paper "Pixel Difference Networks for Efficient Edge Detection" (Oral).

Sync2Gen Code for ICCV 2021 paper: Scene Synthesis via Uncertainty-Driven Attribute Synchronization

Code for the paper "Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds" (ICCV 2021)

Code release for ICCV 2021 paper "Anticipative Video Transformer"

Comments

request to visualizer

Question regarding Garden of Forking Path Dataset

Question about ETH&UCY Dataset

Reproducible MG-GAN code for the FPD dataset

Releases(1.0)

1.0(Aug 14, 2021)

Owner

Sven

Ἀνατομή is a PyTorch library to analyze representation of neural networks

A keras implementation of ENet (abandoned for the foreseeable future)

Source code for the paper "PLOME: Pre-training with Misspelled Knowledge for Chinese Spelling Correction" in ACL2021

Distinguishing Commercial from Editorial Content in News

Code for Neurips2021 Paper "Topology-Imbalance Learning for Semi-Supervised Node Classification".

Oriented Response Networks, in CVPR 2017

Developing your First ML Workflow of the AWS Machine Learning Engineer Nanodegree Program

CLADE - Efficient Semantic Image Synthesis via Class-Adaptive Normalization (TPAMI 2021)

Dewarping Document Image By Displacement Flow Estimation with Fully Convolutional Network.

Code for classifying international patents based on the text of their titles/abstracts

Fast and robust certifiable relative pose estimation

User-friendly bulk RNAseq deconvolution using simulated annealing

RMTD: Robust Moving Target Defence Against False Data Injection Attacks in Power Grids

Official repository for the NeurIPS 2021 paper Get Fooled for the Right Reason: Improving Adversarial Robustness through a Teacher-guided curriculum Learning Approach

Finding Biological Plausibility for Adversarially Robust Features via Metameric Tasks

Pretraining Representations For Data-Efficient Reinforcement Learning

Codes for "CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation"

Age Progression/Regression by Conditional Adversarial Autoencoder

N-RPG - Novel role playing game da turfu

Lingvo is a framework for building neural networks in Tensorflow, particularly sequence models.