Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
File-sharing-Bot: Telegram Bot to store Posts and Documents and it can Access by Special Links.

Bromélia HSS bromelia-hss is the second official implementation of a Diameter-based protocol application by using the Python micro framework Bromélia.

1 Dec 17, 2021
A module grouping multiple translation APIs

translatepy (originally: translate) An aggregation of multiple translation API Translate, transliterate, get the language of texts in no time with the

349 Jan 06, 2023
Total time of all YouTube videos in a playlist.

Youtube Playlist Total Times Total time of all YouTube videos in a playlist. How to Use Download chromedriver depending on your os and chrome version

Mohammad Dori 3 Jul 15, 2022
See trending stock tickers on Reddit and check Stock perfomance

See trending stock tickers on Reddit and check Stock perfomance

Abbas 1.5k Jan 06, 2023
This project is a basic login system in terminal for Discord

Welcome to Discord Login System(Terminal) 👋 This project is a basic login system in terminal for Discord Author 👤 arukovic Github: @SONIC-CODEZ Show

SONIC-CODEZ 2 Feb 11, 2022
A telegram bot to read RSS feeds

Telegram bot to fetch RSS feeds This is a telegram bot that fetches RSS feeds in regular intervals and send it to you. The feed sources can be added o

Santhosh Thottingal 14 Dec 15, 2022
An API which returns random AOT quote everytime it's invoked

An API which returns random AOT quote everytime it's invoked

Nishant Sapkota 1 Feb 07, 2022
A modern, easy to use, feature-rich, and async ready API wrapper for Discord written in Python.

A modern, easy to use, feature-rich, and async ready API wrapper for Discord written in Python. Key Features Modern Pythonic API using async and await

Senpai Development 4 Nov 05, 2021
An API wrapper for convertio.co written in Python.

An API wrapper for convertio.co written in Python.

Moonrise 9 Sep 27, 2022
A python bot that scrapes free udemy coupons and sends them to Telegram.

About: A python telegram bot that scrapes information about fresh free Udemy coupons content from couponscorpion.com and sends it to teleram channel h

Irina Gayday 1 Dec 19, 2021
Clash of Clans developer unofficial api Wrapper to generate ip based token

Clash of Clans developer unofficial api Wrapper to generate ip based token

Aryan Vikash 6 Apr 01, 2022
A code that can make an account bump your discord server 24/7!

BumpCord A code that can make an account bump your discord server 24/7! The main.py is the main file. keep_alive.py prevents your repl from going to s

Phantom 28 Aug 20, 2022
A self-bot for discord, written in Python, which will send you notifications to your desktop if it detects an intruder on your discord server

A self-bot for discord, written in Python, which will send you notifications to your desktop if it detects an intruder on your discord server

LevPrav 1 Jan 11, 2022
Python3 script to dump employee information from XING API

XingDumper Python 3 script to dump company employees from XING API. Perfect OSINT tool ;-) The results contain firstname, lastname, position, gender,

LRVT 11 Dec 26, 2022
A multi-password‌ cracking tool that can help you hack facebook accounts very quickly

FbCracker This is a multi-password‌ cracking tool that can help you hack facebook accounts very quickly. Facebook Hacking Tool Installation On Termux

ReD H4CkeR 9 Nov 16, 2022
An Unofficial API for 1337x, Piratebay, Nyaasi, Torlock, Torrent Galaxy, Zooqle, Kickass, Bitsearch, and MagnetDL

An Unofficial API for 1337x, Piratebay, Nyaasi, Torlock, Torrent Galaxy, Zooqle, Kickass, Bitsearch, and MagnetDL

Neeraj Kumar 130 Dec 27, 2022
A reddit bot that imitates the popular reddit bot "u/repostsleuthbot" to trick people into clicking on a rickroll

Reddit-Rickroll-Bot A reddit bot that imitates the popular reddit bot "u/repostsleuthbot" to trick people into clicking on a rickroll Made with The Py

0 Jul 16, 2022
A library that allows you to easily mock out tests based on AWS infrastructure.

Moto - Mock AWS Services Install $ pip install moto[ec2,s3,all] In a nutshell Moto is a library that allows your tests to easily mock out AWS Services

Steve Pulec 6.5k Jan 02, 2023
Official API documentation for Highrise

Highrise API The Highrise API is implemented as vanilla XML over HTTP using all four verbs (GET/POST/PUT/DELETE). Every resource, like Person, Deal, o

Basecamp 128 Dec 06, 2022
A script to forward mass number of media to another group/channel. Heroku deploy

Telegram Forward Script 😇 This is a Script to Forward Large Number of Files to Another Telegram Channel. Star එකක් දාල fork එකක් ගහපියව් 🥴 If You Tr

Anjana Madu 17 Oct 21, 2022