Multiview Dataset Toolkit

Overview

Multiview Dataset Toolkit

  • Using multi-view cameras is a natural way to obtain a complete point cloud. However, there is to date only one multi-view 3D hand pose dataset– NYU. Furthermore, NYU is primarily used as a depth map dataset; although they also provided the RGB images, these RGB images are of low resolution and quality. FreiHand also records data using a multi- view setup, but the released images are not from corresponding viewpoints. In that sense, it can be regarded only as a single-view dataset containing multiple views rather than a true multi-view dataset.
  • To fill this gap, we present a new multi-view RGB-D 3D hand pose dataset. We use four RealSense D415 cameras in different views to record 4 RGB-D sequences from 4 subjects and the resolution of our recorded dataset is 640 × 480. We use a 21-joint model to annotate the hand pose. Additionally, we provide hand masks, 2D and 3D joint locations, hand meshes in the form of MANO parameters, real complete hand point clouds and full camera parameters. In particular, we provide extrinsic camera parameters so it is easy for users to use multi-view information.

Basic setup

  • download data
  • install basic requirements
pip install numpy matplotlib scikit-image transforms3d tqdm opencv-python trimesh pyrender
  • example code
python toolkit.py

Provided data

  • four views color images
  • four views depth images
  • intrinsic and extrinsic camera parameters
  • 21 hand joints
    • 0 wrist
    • 1 mcp index, 2 pip index, 3 dip index, 4 tip index
    • 5 mcp middle, 6 pip middle, 7 dip middle, 8 tip middle
    • 9 mcp ring, 10 pip ring, 11 dip ring, 12 tip ring
    • 13 mcp pinky, 14 pip pinky, 15 dip pinky, 16 tip pinky
    • 17 mcp thumb, 18 pip thumb, 19 dip thumb, 20 tip thumb
  • mano parameters
    image

Access the dataset

  • data usage in toolkit.py
    • drawMesh
    • drawPose4view
    • getBetterDepth

Info for our camera calibration

Terms of use

@InProceedings{Local2021,
  author    = {Ziwei Yu, Linlin Yang, Shicheng Chen, Angela Yao},
  title     = {Local and Global Point Cloud Reconstruction for 3D Hand Pose Estimation},
  booktitle    = {British Machine Vision Conference (BMVC)},
  year      = {2021},
  url          = {"https://github.com/ShichengChen/multiviewDataset"}
}
Official PyTorch implementation of "The Center of Attention: Center-Keypoint Grouping via Attention for Multi-Person Pose Estimation" (ICCV 21).

CenterGroup This the official implementation of our ICCV 2021 paper The Center of Attention: Center-Keypoint Grouping via Attention for Multi-Person P

Dynamic Vision and Learning Group 43 Dec 25, 2022
3D Pose Estimation for Vehicles

3D Pose Estimation for Vehicles Introduction This work generates 4 key-points and 2 key-edges from vertices and edges of vehicles as ground truth. The

Jingyi Wang 1 Nov 01, 2021
Learning Logic Rules for Document-Level Relation Extraction

LogiRE Learning Logic Rules for Document-Level Relation Extraction We propose to introduce logic rules to tackle the challenges of doc-level RE. Equip

41 Dec 26, 2022
Code for "Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations"

Infinitely Deep Bayesian Neural Networks with SDEs This library contains JAX and Pytorch implementations of neural ODEs and Bayesian layers for stocha

Winnie Xu 95 Nov 26, 2021
Continual reinforcement learning baselines: experiment specifications, implementation of existing methods, and common metrics. Easily extensible to new methods.

Continual Reinforcement Learning This repository provides a simple way to run continual reinforcement learning experiments in PyTorch, including evalu

55 Dec 24, 2022
🔥🔥High-Performance Face Recognition Library on PaddlePaddle & PyTorch🔥🔥

face.evoLVe: High-Performance Face Recognition Library based on PaddlePaddle & PyTorch Evolve to be more comprehensive, effective and efficient for fa

Zhao Jian 3.1k Jan 02, 2023
TorchMultimodal is a PyTorch library for training state-of-the-art multimodal multi-task models at scale.

TorchMultimodal (Alpha Release) Introduction TorchMultimodal is a PyTorch library for training state-of-the-art multimodal multi-task models at scale.

Meta Research 663 Jan 06, 2023
DIR-GNN - Discovering Invariant Rationales for Graph Neural Networks

DIR-GNN "Discovering Invariant Rationales for Graph Neural Networks" (ICLR 2022)

Ying-Xin (Shirley) Wu 70 Nov 13, 2022
PyTorch version of Stable Baselines, reliable implementations of reinforcement learning algorithms.

PyTorch version of Stable Baselines, reliable implementations of reinforcement learning algorithms.

DLR-RM 4.7k Jan 01, 2023
A simple API wrapper for Discord interactions.

Your ultimate Discord interactions library for discord.py. About | Installation | Examples | Discord | PyPI About What is discord-py-interactions? dis

james 641 Jan 03, 2023
An LSTM based GAN for Human motion synthesis

GAN-motion-Prediction An LSTM based GAN for motion synthesis has a few issues reading H3.6M data from A.Jain et al , will fix soon. Prediction of the

Amogh Adishesha 9 Jun 17, 2022
StudioGAN is a Pytorch library providing implementations of representative Generative Adversarial Networks (GANs) for conditional/unconditional image generation.

StudioGAN is a Pytorch library providing implementations of representative Generative Adversarial Networks (GANs) for conditional/unconditional image generation.

3k Jan 08, 2023
pytorch implementation of trDesign

trdesign-pytorch This repository is a PyTorch implementation of the trDesign paper based on the official TensorFlow implementation. The initial port o

Learn Ventures Inc. 41 Dec 29, 2022
Unofficial implementation of Point-Unet: A Context-Aware Point-Based Neural Network for Volumetric Segmentation

Point-Unet This is an unofficial implementation of the MICCAI 2021 paper Point-Unet: A Context-Aware Point-Based Neural Network for Volumetric Segment

Namt0d 9 Dec 07, 2022
Implementation of Neural Style Transfer in Pytorch

PytorchNeuralStyleTransfer Code to run Neural Style Transfer from our paper Image Style Transfer Using Convolutional Neural Networks. Also includes co

Leon Gatys 396 Dec 01, 2022
EfficientNetV2-with-TPU - Cifar-10 case study

EfficientNetV2-with-TPU EfficientNet EfficientNetV2 adalah jenis jaringan saraf convolutional yang memiliki kecepatan pelatihan lebih cepat dan efisie

Sultan syach 1 Dec 28, 2021
Optimized code based on M2 for faster image captioning training

Transformer Captioning This repository contains the code for Transformer-based image captioning. Based on meshed-memory-transformer, we further optimi

lyricpoem 16 Dec 16, 2022
Code Repository for The Kaggle Book, Published by Packt Publishing

The Kaggle Book Data analysis and machine learning for competitive data science Code Repository for The Kaggle Book, Published by Packt Publishing "Lu

Packt 1.6k Jan 07, 2023
A PyTorch implementation of a Factorization Machine module in cython.

fmpytorch A library for factorization machines in pytorch. A factorization machine is like a linear model, except multiplicative interaction terms bet

Jack Hessel 167 Jul 06, 2022
A library for uncertainty quantification based on PyTorch

Torchuq [logo here] TorchUQ is an extensive library for uncertainty quantification (UQ) based on pytorch. TorchUQ currently supports 10 representation

TorchUQ 96 Dec 12, 2022