Code for "Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo"

Last update: Jan 04, 2023

Related tags

Overview

Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo

This repository includes the source code for our CVPR 2021 paper on multi-view multi-person 3D pose estimation. Please read our paper for more details at https://arxiv.org/abs/2104.02273. The project webpage is available here.

Bibtex:

@InProceedings{Lin_2021_CVPR,
    author    = {Lin, Jiahao and Lee, Gim Hee},
    title     = {Multi-View Multi-Person 3D Pose Estimation With Plane Sweep Stereo},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2021},
    pages     = {11886-11895}
}

Environment

Our code is tested on

Python 3.8.5
PyTorch 1.6.0 & torchvision 0.7.0
CUDA 11.2

Preparing Data

Download following data before using the code in this repository:

Annotations and 2D pose predictions for the Campus and the Shelf datasets can be downloaded here. Credit to VoxelPose.
Follow the instructions on the CMU Panoptic Github repo to download the annotations. 2D pose predictions can be downloaded here.
Pre-trained models can be downloaded here.

The data should be organized as follows:

    ROOTDIR/
        └── data/
            └── Campus/
                └── actorsGT.mat
                └── calibration_campus.json
                └── pred_campus_maskrcnn_hrnet_coco.pkl
            └── Shelf/
                └── actorsGT.mat
                └── calibration_shelf.json
                └── pred_shelf_maskrcnn_hrnet_coco.pkl
            └── Panoptic/
                └── 160224_haggling1/
                └── 160226_haggling1/
                └── ...
                └── keypoints_train_results.json
                └── keypoints_validation_results.json
            └── panoptic_training_pose.pkl
        └── output/
            └── campus_synthetic/mvmppe/config/model_best_pretrained.pth.tar
            └── shelf_synthetic/mvmppe/config/model_best_pretrained.pth.tar
            └── panoptic/mvmppe/config/model_best_pretrained.pth.tar
        └── ...

Training and Inference

Below are the commands for training our model on different datasets.

The Campus dataset:

    python run/train.py --cfg configs/campus/config.yaml

The Shelf dataset:

    python run/train.py --cfg configs/shelf/config.yaml

The CMU Panoptic dataset:

    python run/train.py --cfg configs/panoptic/config.yaml

Below are the commands for performing inference with our pre-trained models.

The Campus dataset:

    python run/validate.py --cfg configs/campus/config.yaml -t pretrained

The Shelf dataset:

    python run/validate.py --cfg configs/shelf/config.yaml -t pretrained

The CMU Panoptic dataset:

    python run/validate.py --cfg configs/panoptic/config.yaml -t pretrained

Code for "Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo"

Related tags

Overview

Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo

Environment

Preparing Data

Training and Inference

Owner

Jiahao Lin

A PyTorch implementation of the Relational Graph Convolutional Network (RGCN).

YoloV5 implemented by TensorFlow2 , with support for training, evaluation and inference.

The Official Implementation of the ICCV-2021 Paper: Semantically Coherent Out-of-Distribution Detection.

Code, Models and Datasets for OpenViDial Dataset

Multi-objective gym environments for reinforcement learning.

Extracting knowledge graphs from language models as a diagnostic benchmark of model performance.

Simple Tensorflow implementation of "Adaptive Convolutions for Structure-Aware Style Transfer" (CVPR 2021)

Code for "Finding Regions of Heterogeneity in Decision-Making via Expected Conditional Covariance" at NeurIPS 2021

The implement of papar "Enhanced Graph Learning for Collaborative Filtering via Mutual Information Maximization"

Source code, data, and evaluation details for “Cross-Lingual Citations in English Papers: A Large-Scale Analysis of Prevalence, Formation, and Ramifications”

Implement A3C for Mujoco gym envs

PyTorch code for MART: Memory-Augmented Recurrent Transformer for Coherent Video Paragraph Captioning

DeepLab is a state-of-art deep learning system for semantic image segmentation built on top of Caffe.

Deepface is a lightweight face recognition and facial attribute analysis (age, gender, emotion and race) framework for python

efficient neural audio synthesis in the waveform domain

End-to-end face detection, cropping, norm estimation, and landmark detection in a single onnx model

Official implementation of AAAI-21 paper "Label Confusion Learning to Enhance Text Classification Models"

Code for WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models.

Official pytorch code for SSC-GAN: Semi-Supervised Single-Stage Controllable GANs for Conditional Fine-Grained Image Generation(ICCV 2021)

なりすまし検出(anti-spoof-mn3)のWebカメラ向けデモ