Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
SongLink Discord Bot - Discord bot to share music links easily

SongLink_Discord_Bot Discord bot to share music links easily. Take a link from y

Edgar Lefevre 4 Feb 18, 2022
This library is for simplified work with the sms-man.com API

SMSMAN Public API Python This is a lightweight library that works as a connector to Sms-Man public API Installation pip install smsman Documentation h

13 Nov 19, 2022
This tool helps users selecting items from the Gwennen gambling trade (based on prices of the uniques).

Gwennen Gambler This small program will check each item in the Gwennen shop (item gamble) according and show small stats according to poe.ninja. Shoul

9 Apr 10, 2022
An simple python script for remove rockstar account for fivem, very useful for cheating or change account for unban from an server, remember change steam account.

This tool is used for be unbanned from servers, u need disconnect the discord, use other steam account and uninstall xbox for be unbanned 100%, it only work for unban in server, not global

MiguDEV 4 Oct 10, 2022
CnCL - CnCLess it's an Easy to deploy Botnet without CnC/C2

CnCL CnCLess it's an Easy to deploy Botnet without CnC/C2, Harder to track and t

ZSendokame 2 Jan 10, 2022
A Python library for the Buildkite API

PyBuildkite A Python library and client for the Buildkite API. Usage To get the package, execute: pip install pybuildkite Then set up an instance of

Peter Yasi 29 Nov 30, 2022
Programmeertheorie 2022 - Team Trainspotters - RailNL

Trainspotters Vak: Programmeertheorie 2022 Gekozen case: RailNL Teamnaam: Trainspotters Studenten: Mijntje Meijer, Sam Bijhouwer, Maik Larooij To-do's

Maik Larooij 1 Jan 25, 2022
Powerful Telegram bot to countdown to your important events in any group chat.

Powerful Telegram bot to countdown to your important events in any group chat. Live countdown timer.

118 Dec 30, 2022
Auxiliator is telegram bot for basic web-application analysis

Auxiliator Auxiliator is telegram bot for basic web-application analysis What for? Sometimes there is no access to your main PC, where you can scan we

Revoltage 13 Dec 26, 2021
A Python API for Connected 2

connected API for Connected 2 api for the { connected 2 } programmer : api report api follow api check username api forget password api Search api cha

2 Jun 05, 2022
This project checks the weather in the next 12 hours and sends an SMS to your phone number if it's going to rain to remind you to take your umbrella.

RainAlert-Request-Twilio This project checks the weather in the next 12 hours and sends an SMS to your phone number if it's going to rain to remind yo

9 Apr 15, 2022
👨‍💼Linkedin API for Python

linkedin_api 👨‍💼 Linkedin API for Python No "official" API access required - just use a valid Linkedin account! Programmatically send messages, get

Tom Quirk 918 Dec 29, 2022
Easy way to use Telegram bot to hide your identity.

telegram-support-bot Easy way to use Telegram bot to hide your identity. Useful for support, anonymous channel management. Free clone of Livegram Bot.

Daniil Okhlopkov 197 Dec 23, 2022
Discord Token Generator of a project - Some stupids ppl are trying to leak it so i'm leaking faster :)

Original creator: Rolf (dort) HCaptcha Bypasser: h0nde Shark.Solar Discord Token Generator of a project - Some stupids ppl are trying to leak it so i'

Stanley 14 Sep 29, 2021
A discord program that will send a message to nearly every user in a discord server

Discord Mass DM Scrapes users from a discord server to promote/mass dm Report Bug · Request Feature Features Asynchronous Easy to use Free Auto scrape

dropout 56 Jan 02, 2023
SEP Finder Bot

SEP Finder Bot This is a Telegram bot that will help you find the correct SEP and Baseband files to use for your device with futurerestore. Usage A ho

6 Dec 03, 2022
A self-hosted Discord music bot.

Cassette A self-hosted Discord music bot. Requirements py-cord pynacl pytube Setup Intended to be hosted on Heroku. Fork or clone this repo. Create a

Lohan 8 Apr 28, 2022
Image Tooᥣs Bot I specialize for logo design Services with Amazing logo Creator Platform and more tools

Image Tooᥣs Bot I specialize for logo design Services with Amazing logo Creator Platform and more tools

Sz Team Bots <sz/>✌️ 10 Oct 21, 2022
Ever wanted a dashboard for making your antispam? This is it.

Ever wanted a dashboard for making your antispam? This is it.

Skelmis 1 Oct 27, 2021
Decryption utility for PGP Whole Disk Encryption

wdepy: Decryption and Inspection for PGP WDE Disks This is a small python tool to inspect and decrypt disk images encrypted with PGP Whole Disk Encryp

Brendan Dolan-Gavitt 17 Oct 07, 2022