This is the official code of Rethinking the Heatmap Regression for Bottom-up Human Pose Estimation. This paper has been accepted to CVPR2021.
This repo is built on Bottom-up-Higher-HRNet.
| Method | Backbone | Input size | #Params | GFLOPs | AP | Ap .5 | AP .75 | AP (M) | AP (L) |
|---|---|---|---|---|---|---|---|---|---|
| HigherHRNet | HRNet-w32 | 512 | 28.6M | 47.9 | 67.1 | 86.2 | 73.0 | 61.5 | 76.1 |
| HigherHRNet + SWAHR | HRNet-w32 | 512 | 28.6M | 48.0 | 68.9 | 87.8 | 74.9 | 63.0 | 77.4 |
| HigherHRNet | HRNet-w48 | 640 | 63.8M | 154.3 | 69.9 | 87.2 | 76.1 | 65.4 | 76.4 |
| HigherHRNet + SWAHR | HRNet-w48 | 640 | 63.8M | 154.6 | 70.8 | 88.5 | 76.8 | 66.3 | 77.4 |
| Method | Backbone | Input size | #Params | GFLOPs | AP | Ap .5 | AP .75 | AP (M) | AP (L) |
|---|---|---|---|---|---|---|---|---|---|
| HigherHRNet | HRNet-w32 | 512 | 28.6M | 47.9 | 69.9 | 87.1 | 76.0 | 65.3 | 77.0 |
| HigherHRNet + SWAHR | HRNet-w32 | 512 | 28.6M | 48.0 | 71.4 | 88.9 | 77.8 | 66.3 | 78.9 |
| HigherHRNet | HRNet-w48 | 640 | 63.8M | 154.3 | 72.1 | 88.4 | 78.2 | 67.8 | 78.3 |
| HigherHRNet + SWAHR | HRNet-w48 | 640 | 63.8M | 154.6 | 73.2 | 89.8 | 79.1 | 69.1 | 79.3 |
| Method | Backbone | Input size | #Params | GFLOPs | AP | Ap .5 | AP .75 | AP (M) | AP (L) |
|---|---|---|---|---|---|---|---|---|---|
| OpenPose* | - | - | - | - | 61.8 | 84.9 | 67.5 | 57.1 | 68.2 |
| Hourglass | Hourglass | 512 | 277.8M | 206.9 | 56.6 | 81.8 | 61.8 | 49.8 | 67.0 |
| PersonLab | ResNet-152 | 1401 | 68.7M | 405.5 | 66.5 | 88.0 | 72.6 | 62.4 | 72.3 |
| PifPaf | - | - | - | - | 66.7 | - | - | 62.4 | 72.9 |
| Bottom-up HRNet | HRNet-w32 | 512 | 28.5M | 38.9 | 64.1 | 86.3 | 70.4 | 57.4 | 73.9 |
| HigherHRNet | HRNet-w32 | 512 | 28.6M | 47.9 | 66.4 | 87.5 | 72.8 | 61.2 | 74.2 |
| HigherHRNet + SWAHR | HRNet-w32 | 512 | 28.6M | 48.0 | 67.9 | 88.9 | 74.5 | 62.4 | 75.5 |
| HigherHRNet | HRNet-w48 | 640 | 63.8M | 154.3 | 68.4 | 88.2 | 75.1 | 64.4 | 74.2 |
| HigherHRNet + SWAHR | HRNet-w48 | 640 | 63.8M | 154.6 | 70.2 | 89.9 | 76.9 | 65.2 | 77.0 |
| Method | Backbone | Input size | #Params | GFLOPs | AP | Ap .5 | AP .75 | AP (M) | AP (L) |
|---|---|---|---|---|---|---|---|---|---|
| Hourglass | Hourglass | 512 | 277.8M | 206.9 | 63.0 | 85.7 | 68.9 | 58.0 | 70.4 |
| Hourglass* | Hourglass | 512 | 277.8M | 206.9 | 65.5 | 86.8 | 72.3 | 60.6 | 72.6 |
| PersonLab | ResNet-152 | 1401 | 68.7M | 405.5 | 68.7 | 89.0 | 75.4 | 64.1 | 75.5 |
| HigherHRNet | HRNet-w48 | 640 | 63.8M | 154.3 | 70.5 | 89.3 | 77.2 | 66.6 | 75.8 |
| HigherHRNet + SWAHR | HRNet-w48 | 640 | 63.8M | 154.6 | 72.0 | 90.7 | 78.8 | 67.8 | 77.7 |
| Method | AP | Ap .5 | AP .75 | AP (E) | AP (M) | AP (H) |
|---|---|---|---|---|---|---|
| Mask-RCNN | 57.2 | 83.5 | 60.3 | 69.4 | 57.9 | 45.8 |
| AlphaPose | 61.0 | 81.3 | 66.0 | 71.2 | 61.4 | 51.1 |
| SPPE | 66.0. | 84.2 | 71.5 | 75.5 | 66.3 | 57.4 |
| OpenPose | - | - | - | 62.7 | 48.7 | 32.3 |
| HigherHRNet | 65.9 | 86.4 | 70.6 | 73.3 | 66.5 | 57.9 |
| HigherHRNet + SWAHR | 71.6 | 88.5 | 77.6 | 78.9 | 72.4 | 63.0 |
| HigherHRNet* | 67.6 | 87.4 | 72.6 | 75.8 | 68.1 | 58.9 |
| HigherHRNet + SWAHR* | 73.8 | 90.5 | 79.9 | 81.2 | 74.7 | 64.7 |
'*' indicates multi-scale test
The details about preparing the environment and datasets can be referred to README.md.
Downlaod our pretrained weights from BaidunYun(Password: 8weh) or GoogleDrive to ./models.
For single-scale testing:
python tools/dist_valid.py \
--cfg experiments/coco/higher_hrnet/w32_512_adam_lr1e-3.yaml \
TEST.MODEL_FILE models/pose_coco/pose_higher_hrnet_w32_512.pth
By default, we use horizontal flip. To test without flip:
python tools/dist_valid.py \
--cfg experiments/coco/higher_hrnet/w32_512_adam_lr1e-3.yaml \
TEST.MODEL_FILE models/pose_coco/pose_higher_hrnet_w32_512.pth \
TEST.FLIP_TEST False
Multi-scale testing is also supported, although we do not report results in our paper:
python tools/dist_valid.py \
--cfg experiments/coco/higher_hrnet/w32_512_adam_lr1e-3.yaml \
TEST.MODEL_FILE models/pose_coco/pose_higher_hrnet_w32_512.pth \
TEST.SCALE_FACTOR '[0.5, 1.0, 2.0]'
python tools/dist_train.py \
--cfg experiments/coco/higher_hrnet/w32_512_adam_lr1e-3.yaml
By default, it will use all available GPUs on the machine for training. To specify GPUs, use
CUDA_VISIBLE_DEVICES=0,1 python tools/dist_train.py \
--cfg experiments/coco/higher_hrnet/w32_512_adam_lr1e-3.yaml
python tools/dist_inference.py \
--img_dir path/to/your/directory/of/images \
--save_dir path/where/results/are/saved \
--cfg experiments/coco/higher_hrnet/w32_512_adam_lr1e-3.yaml \
TEST.MODEL_FILE models/pose_coco/pose_higher_hrnet_w32_512.pth \
TEST.SCALE_FACTOR '[0.5, 1.0, 2.0]'
If you find this work or code is helpful in your research, please cite:
@inproceedings{LuoSWAHR,
title={Rethinking the Heatmap Regression for Bottom-up Human Pose Estimation},
author={Zhengxiong Luo and Zhicheng Wang and Yan Huang and Liang Wang and Tieniu Tan and Erjin Zhou},
booktitle={CVPR},
year={2021}
}