HPRNet: Hierarchical Point Regression for Whole-Body Human Pose Estimation

Related tags

Deep LearningHPRNet
Overview

HPRNet: Hierarchical Point Regression for Whole-Body Human Pose Estimation

Official PyTroch implementation of HPRNet.

HPRNet: Hierarchical Point Regression for Whole-Body Human Pose Estimation,
Nermin Samet, Emre Akbas,
Under review. (arXiv pre-print)

Highlights

  • HPRNet is a bottom-up, one-stage and hierarchical keypoint regression method for whole-body pose estimation.
  • HPRNet has the best performance among bottom-up methods for all the whole-body parts.
  • HPRNet achieves SOTA performance for the face (76.0 AP) and hand (51.2 AP) keypoint estimation.
  • Unlike two-stage methods, HPRNet predicts whole-body pose in a constant time independent of the number of people in an image.

COCO-WholeBody Keypoint Estimation Results

Model Body AP Foot AP Face AP Hand AP Whole-body AP Download
HPRNet (DLA) 55.2 / 57.1 49.1 / 50.7 74.6 / 75.4 47.0 / 48.4 31.5 / 32.7 model
HPRNet (Hourglass) 59.4 / 61.1 53.0 / 53.9 75.4 / 76.0 50.4 / 51.2 34.8 / 34.9 model
  • Results are presented without and with test time flip augmentation respectively.
  • All models are trained on COCO-WholeBody train2017 and evaluated on val2017.
  • The models can be downloaded directly from Google drive.

Installation

  1. [Optional but recommended] create a new conda environment.

    conda create --name HPRNet python=3.7
    

    And activate the environment.

    conda activate HPRNet
    
  2. Clone the repo:

    HPRNet_ROOT=/path/to/clone/HPRNet
    git clone https://github.com/nerminsamet/HPRNet $HPRNet_ROOT
    
  3. Install PyTorch 1.4.0:

    conda install pytorch torchvision cudatoolkit=10.0 -c pytorch
    
  4. Install the requirements:

    pip install -r requirements.txt
    
  5. Compile DCNv2 (Deformable Convolutional Networks):

    cd $HPRNet_ROOT/src/lib/models/networks/DCNv2
    ./make.sh
    

Dataset preparation

  • Download the images (2017 Train, 2017 Val) from coco website.

  • Download train and val annotation files.

    ${COCO_PATH}
    |-- annotations
        |-- coco_wholebody_train_v1.0.json
        |-- coco_wholebody_val_v1.0.json
    |-- images
        |-- train2017
        |-- val2017 
    

Evaluation and Training

  • You could find all the evaluation and training scripts in the experiments folder.
  • For evaluation, please download the pretrained models you want to evaluate and put them in HPRNet_ROOT/models/.
  • In the case that you don't have 4 GPUs, you can follow the linear learning rate rule to adjust the learning rate.
  • If the training is terminated before finishing, you can use the same command with --resume to resume training.

Acknowledgement

The numerical calculations reported in this paper were fully performed at TUBITAK ULAKBIM, High Performance and Grid Computing Center (TRUBA resources).

License

HPRNet is released under the MIT License (refer to the LICENSE file for details).

Citation

If you find HPRNet useful for your research, please cite our paper as follows:

N. Samet, E. Akbas, "HPRNet: Hierarchical Point Regression for Whole-Body Human Pose Estimation", arXiv, 2021.

BibTeX entry:

@misc{hprnet,
      title={HPRNet: Hierarchical Point Regression for Whole-Body Human Pose Estimation}, 
      author={Nermin Samet and Emre Akbas},
      year={2021}, 
}
Owner
Nermin Samet
PhD candidate
Nermin Samet
Awesome-AI-books - Some awesome AI related books and pdfs for learning and downloading

Awesome AI books Some awesome AI related books and pdfs for downloading and learning. Preface This repo only used for learning, do not use in business

luckyzhou 1k Jan 01, 2023
working repo for my xumx-sliCQ submissions to the ISMIR 2021 MDX

Music Demixing Challenge - xumx-sliCQ This repository is the GitHub mirror of my working submission repository for the AICrowd ISMIR 2021 Music Demixi

4 Aug 25, 2021
Picasso: A CUDA-based Library for Deep Learning over 3D Meshes

The Picasso Library is intended for complex real-world applications with large-scale surfaces, while it also performs impressively on the small-scale applications over synthetic shape manifolds. We h

97 Dec 01, 2022
Matplotlib Image labeller for classifying images

mpl-image-labeller Use Matplotlib to label images for classification. Works anywhere Matplotlib does - from the notebook to a standalone gui! For more

Ian Hunt-Isaak 5 Sep 24, 2022
Mmdetection3d Noted - MMDetection3D is an open source object detection toolbox based on PyTorch

MMDetection3D is an open source object detection toolbox based on PyTorch

Jiangjingwen 13 Jan 06, 2023
custom pytorch implementation of MoCo v3

MoCov3-pytorch custom implementation of MoCov3 [arxiv]. I made minor modifications based on the official MoCo repository [github]. No ViT part code an

39 Nov 14, 2022
Code repository for the paper "Tracking People with 3D Representations"

Tracking People with 3D Representations Code repository for the paper "Tracking People with 3D Representations" (paper link) (project site). Jathushan

Jathushan Rajasegaran 77 Dec 03, 2022
TensorFlow2 Classification Model Zoo playing with TensorFlow2 on the CIFAR-10 dataset.

Training CIFAR-10 with TensorFlow2(TF2) TensorFlow2 Classification Model Zoo. I'm playing with TensorFlow2 on the CIFAR-10 dataset. Architectures LeNe

Chia-Hung Yuan 16 Sep 27, 2022
CCP dataset from Clothing Co-Parsing by Joint Image Segmentation and Labeling

Clothing Co-Parsing (CCP) Dataset Clothing Co-Parsing (CCP) dataset is a new clothing database including elaborately annotated clothing items. 2, 098

Wei Yang 434 Dec 24, 2022
GeDML is an easy-to-use generalized deep metric learning library

GeDML is an easy-to-use generalized deep metric learning library

Borui Zhang 32 Dec 05, 2022
A project studying the influence of communication in multi-objective normal-form games

Communication in Multi-Objective Normal-Form Games This repo consists of five different types of agents that we have used in our study of communicatio

Willem RΓΆpke 0 Dec 17, 2021
MatchGAN: A Self-supervised Semi-supervised Conditional Generative Adversarial Network

MatchGAN: A Self-supervised Semi-supervised Conditional Generative Adversarial Network This repository is the official implementation of MatchGAN: A S

Justin Sun 12 Dec 27, 2022
πŸ”₯ Cannlytics-powered artificial intelligence πŸ€–

Cannlytics AI πŸ”₯ Cannlytics-powered artificial intelligence πŸ€– πŸ—οΈ Installation πŸƒβ€β™€οΈ Quickstart 🧱 Development 🦾 Automation πŸ’Έ Support πŸ›οΈ License ?

Cannlytics 3 Nov 11, 2022
Real Time Object Detection and Classification using Yolo Algorithm.

Real time Object detection & Classification using YOLO algorithm. Real Time Object Detection and Classification using Yolo Algorithm. What is Object D

Ketan Chawla 1 Apr 17, 2022
Graph InfoClust: Leveraging cluster-level node information for unsupervised graph representation learning

Graph-InfoClust-GIC [PAKDD 2021] PAKDD'21 version Graph InfoClust: Maximizing Coarse-Grain Mutual Information in Graphs Preprint version Graph InfoClu

Costas Mavromatis 21 Dec 03, 2022
This is the official Pytorch implementation of "Lung Segmentation from Chest X-rays using Variational Data Imputation", Raghavendra Selvan et al. 2020

README This is the official Pytorch implementation of "Lung Segmentation from Chest X-rays using Variational Data Imputation", Raghavendra Selvan et a

Raghav 42 Dec 15, 2022
TAP: Text-Aware Pre-training for Text-VQA and Text-Caption, CVPR 2021 (Oral)

TAP: Text-Aware Pre-training TAP: Text-Aware Pre-training for Text-VQA and Text-Caption by Zhengyuan Yang, Yijuan Lu, Jianfeng Wang, Xi Yin, Dinei Flo

Microsoft 61 Nov 14, 2022
A PyTorch implementation of "ANEMONE: Graph Anomaly Detection with Multi-Scale Contrastive Learning", CIKM-21

ANEMONE A PyTorch implementation of "ANEMONE: Graph Anomaly Detection with Multi-Scale Contrastive Learning", CIKM-21 Dependencies python==3.6.1 dgl==

Graph Analysis & Deep Learning Laboratory, GRAND 30 Dec 14, 2022
Jarvis Project is a basic virtual assistant that uses TensorFlow for learning.

Jarvis_proyect Jarvis Project is a basic virtual assistant that uses TensorFlow for learning. Latest version 0.1 Features: Good morning protocol Tell

Anze Kovac 3 Aug 31, 2022
The implementation of the CVPR2021 paper "Structure-Aware Face Clustering on a Large-Scale Graph with 10^7 Nodes"

STAR-FC This code is the implementation for the CVPR 2021 paper "Structure-Aware Face Clustering on a Large-Scale Graph with 10^7 Nodes" 🌟 🌟 . πŸŽ“ Re

Shuai Shen 87 Dec 28, 2022