Ultra-lightweight human body posture key point CNN model. ModelSize:2.3MB HUAWEI P40 NCNN benchmark: 6ms/img,

Last update: Dec 27, 2022

Overview

Ultralight-SimplePose

Support NCNN mobile terminal deployment
Based on MXNET(>=1.5.1) GLUON(>=0.7.0) framework
Top-down strategy: The input image is the person ROI detected by the object detector
Lightweight mobile terminal human body posture key point model(COCO 17 person_keypoints)
Detector:https://github.com/dog-qiuqiu/MobileNetv2-YOLOV3

Model

Mobile inference frameworks benchmark (4*ARM_CPU)

Network	Resolution	Inference time (NCNN/Kirin 990)	FLOPS	Weight size	HeatmapAccuracy
Ultralight-Nano-SimplePose	W:192 H:256	~5.4ms	0.224BFlops	2.3MB	74.3%

COCO2017 val keypoints metrics evaluate

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets= 20 ] = 0.518
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets= 20 ] = 0.816
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets= 20 ] = 0.558
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.498
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.549
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 20 ] = 0.563
 Average Recall     (AR) @[ IoU=0.50      | area=   all | maxDets= 20 ] = 0.837
 Average Recall     (AR) @[ IoU=0.75      | area=   all | maxDets= 20 ] = 0.607
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.535
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.604

Install

pip install mxnet-cu101 gluoncv
pip install opencv-python cython pycocotools

Install mxnet according to your own cuda version

Demo

Test picture

python img_demo.py

Test camera stream

python cam_demo

How To Train

Download the coco2017 dataset

http://images.cocodataset.org/zips/train2017.zip
http://images.cocodataset.org/annotations/annotations_trainval2017.zip
http://images.cocodataset.org/zips/val2017.zip
Unzip the downloaded dataset zip file to the coco directory
交流qq群:1062122604

Train

python train_simple_pose.py

Ncnn Deploy

Dependent library: Opencv Ncnn
Read the camera video stream test by default, if you test the picture, please modify the code

Install ncnn

$ git clone https://github.com/Tencent/ncnn.git
$ cd <ncnn-root-dir>
$ mkdir -p build
$ cd build
$ make -j4
$ make install

Run ncnn sample

$ cp -rf ncnn/build/install/include ./Ultralight-SimplePose/ncnnsample/
$ cp -rf ncnn/build/install/lib ./Ultralight-SimplePose/ncnnsample/
$ g++ -o ncnnpose ncnnpose.cpp -I include/ncnn/ lib/libncnn.a `pkg-config --libs --cflags opencv` -fopenmp
$ ./ncnnpose

Ultra-lightweight human body posture key point CNN model. ModelSize:2.3MB HUAWEI P40 NCNN benchmark: 6ms/img,

Related tags

Overview

Ultralight-SimplePose

Model

Mobile inference frameworks benchmark (4*ARM_CPU)

COCO2017 val keypoints metrics evaluate

Install

Demo

Test picture

Test camera stream

How To Train

Download the coco2017 dataset

Train

Ncnn Deploy

Install ncnn

Run ncnn sample

Ncnn Picture test results

Android sample

Thanks

Owner

Code & Experiments for "LILA: Language-Informed Latent Actions" to be presented at the Conference on Robot Learning (CoRL) 2021.

Text Summarization - WCN — Weighted Contextual N-gram method for evaluation of Text Summarization

Homepage of paper: Paint Transformer: Feed Forward Neural Painting with Stroke Prediction, ICCV 2021.

"Domain Adaptive Semantic Segmentation without Source Data" (ACM MM 2021)

In this tutorial, you will perform inference across 10 well-known pre-trained object detectors and fine-tune on a custom dataset. Design and train your own object detector.

A python3 tool to take a 360 degree survey of the RF spectrum (hamlib + rotctld + RTL-SDR/HackRF)

[Machine Learning Engineer Basic Guide] 부스트캠프 AI Tech - Product Serving 자료

Contextualized Perturbation for Textual Adversarial Attack, NAACL 2021

A JAX-based research framework for writing differentiable numerical simulators with arbitrary discretizations

An NLP library with Awesome pre-trained Transformer models and easy-to-use interface, supporting wide-range of NLP tasks from research to industrial applications.

Fast Style Transfer in TensorFlow

Lazy, a tool for running things in idle time

[NeurIPS 2021] Source code for the paper "Qu-ANTI-zation: Exploiting Neural Network Quantization for Achieving Adversarial Outcomes"

Evolving neural network parameters in JAX.

ZeroVL - The official implementation of ZeroVL

DSEE: Dually Sparsity-embedded Efficient Tuning of Pre-trained Language Models

A Closer Look at Invalid Action Masking in Policy Gradient Algorithms

Official PyTorch implementation of "Preemptive Image Robustification for Protecting Users against Man-in-the-Middle Adversarial Attacks" (AAAI 2022)

A Python implementation of global optimization with gaussian processes.

DyNet: The Dynamic Neural Network Toolkit