Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking

Last update: Dec 12, 2022

Related tags

Deep Learning ArTIST

Overview

Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking (CVPR 2021)

Pytorch implementation of the ArTIST motion model. In this repo, there are

Training script for the Moving Agent network
Training script for the ArTIST motion model
Demo script for Inferring the likelihood of current observations (detections)
Demo script for Inpainting the missing observation/detections

Demo 1: Likelihood estimation of observation

Run:

python3 demo_scoring.py

This will generate the output in the temp/ar/log_p directory, look like this:

This demo gets as input a pretrained model of the Moving Agent Network (MA-Net), a pretrained model of ArTIST, the centroids (obtain centroids via the script in the utils), a demo test sample index and the number of clusters.

The model then evaluates the log-likelihood (lower the better) of all detections as the continuation of the observed sequence.

Demo 2: Sequence inpainting

Run:

python3 demo_inpainting.py

This will generate the multiple plauusible continuations of an observed motion, stored in the temp/ar/inpainting directory. One example looks like this:

For each generated future sequence, it computes the IoU between the last generated bounding box and the last groundtruth bounding box, as well as the mean IoU for the entire generated sequence and the groundtruth sequence.

Utilities

In this repo, there are a number of scripts to generate the required data to train/evaluate ArTIST.

prepare_data: Given the annotations of a dataset (e.g., MOT17), it extracts the motion sequences as well as the IDs of the social tracklets living the life span of the corresponding sequence, and stores it as a dictionary. If there are multiple tracking datasets that you wish to combine, you can use the merge_datasets() function inside this script.
clustering: Given the output dictionary of prepare_data script, this script performs the K-Means clustering and stores the centroids which are then used in the ArTIST model.
dataloader_ae and dataloader_ar: Given the post-processes version of the dataset dictionary (which can be done by running the post_process script), these two scripts define the dataloaders for training the MA-Net and ArTIST. Note that the dataloader of ArTIST uses the MA-Net to compute the social information. This can also be done jointly in an end-to-end fashion, which we observed almost no difference.
create_demo_test_subset: In order to run the demo scripts, you need to run this script. However, the demo test subset has been produced and stored in data/demo_test_subset.npy.

Data

You can download the required data from the Release and put it in data/ directory.

Citation

If you find this work useful in your own research, please consider citing:

@inproceedings{saleh2021probabilistic,
author={Saleh, Fatemeh and Aliakbarian, Sadegh and Rezatofighi, Hamid and Salzmann, Mathieu and Gould, Stephen},
title = {Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking},
booktitle={Proceedings of the IEEE/CVF conference on computer vision and pattern recognition},
year = {2021}
}

Multiple Object Tracking with Yolov5!

Tracking with yolov5 This implementation is for who need to tracking multi-object only with detector. You can easily track mult-object with your well

9 Nov 8, 2022

A New Approach to Overgenerating and Scoring Abstractive Summaries

We provide the source code for the paper "A New Approach to Overgenerating and Scoring Abstractive Summaries" accepted at NAACL'21. If you find the code useful, please cite the following paper.

4 Apr 3, 2022

Y. Zhang, Q. Yao, W. Dai, L. Chen. AutoSF: Searching Scoring Functions for Knowledge Graph Embedding. IEEE International Conference on Data Engineering (ICDE). 2020

AutoSF The code for our paper "AutoSF: Searching Scoring Functions for Knowledge Graph Embedding" and this paper has been accepted by ICDE2020. News:

64 Dec 17, 2022

SiamMOT is a region-based Siamese Multi-Object Tracking network that detects and associates object instances simultaneously.

SiamMOT: Siamese Multi-Object Tracking

432 Dec 17, 2022

Comments

Re-creating paper results

Did you use implement the ArTIST paradigm in the SORT algorithm to achieve the results in your paper? If so, do you have an example of integrating the ArTIST motion model with SORT? I am trying to re-create the results of the paper.

How do I re-create the results you obtained in your paper?

opened by vineetrshenoy 1
dataloader.py: shape mismatch

when i use dataloader.py to load the data, here comes a error:could not broadcast input array from shape (2) into shape (4) in line 33 of dataloader.py, I wonder how to fix the bug and what is the data format in data/postp_combined_path_mot_train.npy, thanks for your help.

opened by guileihu 0

Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking

Related tags

Overview

Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking (CVPR 2021)

Demo 1: Likelihood estimation of observation

Demo 2: Sequence inpainting

Utilities

Data

Citation

You might also like...

Multiple Object Tracking with Yolov5!

A New Approach to Overgenerating and Scoring Abstractive Summaries

Y. Zhang, Q. Yao, W. Dai, L. Chen. AutoSF: Searching Scoring Functions for Knowledge Graph Embedding. IEEE International Conference on Data Engineering (ICDE). 2020

The code for our paper "AutoSF: Searching Scoring Functions for Knowledge Graph Embedding"

Image-popularity-score - A novel deep regression method for image scoring.

Object tracking and object detection is applied to track golf puts in real time and display stats/games.

Official PyTorch implementation of Joint Object Detection and Multi-Object Tracking with Graph Neural Networks

Object Detection and Multi-Object Tracking

SiamMOT is a region-based Siamese Multi-Object Tracking network that detects and associates object instances simultaneously.

Comments

Re-creating paper results

dataloader.py: shape mismatch

Releases(data-release)

data-release(May 9, 2021)

Owner

Fatemeh

The official repository for "Intermediate Layers Matter in Momentum Contrastive Self Supervised Learning" paper.

Measuring if attention is explanation with ROAR

PyTorch implementation of Off-policy Learning in Two-stage Recommender Systems

A self-supervised 3D representation learning framework named viewpoint bottleneck.

PyTorch implementation of "A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancement."

Implementation of Retrieval-Augmented Denoising Diffusion Probabilistic Models in Pytorch

Tensorflow implementation and notebooks for Implicit Maximum Likelihood Estimation

The datasets and code of ACL 2021 paper "Aspect-Category-Opinion-Sentiment Quadruple Extraction with Implicit Aspects and Opinions".

Code release for Hu et al. Segmentation from Natural Language Expressions. in ECCV, 2016

Compositional and Parameter-Efficient Representations for Large Knowledge Graphs

EZ graph is an easy to use AI solution that allows you to make and train your neural networks without a single line of code.

3D AffordanceNet is a 3D point cloud benchmark consisting of 23k shapes from 23 semantic object categories, annotated with 56k affordance annotations and covering 18 visual affordance categories.

Implementation of Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning

The PyTorch improved version of TPAMI 2017 paper: Face Alignment in Full Pose Range: A 3D Total Solution.

Prompts - Read a textfile of prompts and import into anki via ankiconnect

2D Time independent Schrodinger equation solver for arbitrary shape of well

💊 A 3D Generative Model for Structure-Based Drug Design (NeurIPS 2021)

GNNAdvisor: An Efficient Runtime System for GNN Acceleration on GPUs

Implementation of the ICCV'21 paper Temporally-Coherent Surface Reconstruction via Metric-Consistent Atlases

Alfred-Restore-Iterm-Arrangement - An Alfred workflow to restore iTerm2 window Arrangements