Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
Sniper for Anigame and Izzi discord bots!

Anigame Sniper Gen-3 Features Inbuilt Spammer Responds to your messages in discord (on/off) Snipes only where you want it to Set latency so that the b

22 Nov 13, 2022
The world's first public V2ray manager Telegram bot

πŸ“Œ DarkV2ray-Manager-Bot 0.1 UPDATE 11/11/2021 Telegram bot v2ray Test user expired date data limit paylode && sni usage user on/off heroku bot hostin

@Dk_king_offcial 1 Nov 11, 2021
Quack-SMS-BOMBER - Quack Toolkit By IkigaiHack

Quack Toolkit By IkigaiHack About Quack Toolkit Quack Toolkit is a set of tools

Marcel 2 Aug 19, 2022
Deepak Clouds Torrent is a multipurpose Telegram Bot writen in Python for mirroring files on the Internet to our beloved Google Drive.

Deepak Clouds Torrent is a multipurpose Telegram Bot writen in Python for mirroring files on the Internet to our beloved Google Drive.

Deepak Clouds 37 Oct 28, 2022
A Telegram bot to download posts, videos, reels, IGTV and a user profile picture from Instagram!

Telegram Bot A telegram bot to download media from Instagram! No API Key or Login Needed! Requirements You must have python installed (of course) You

Simon Farah 2 Apr 10, 2022
Project for QVault Hackathon which plays sounds based on the letters of a user's name

virtual_instrument Project for QVault Hackathon which plays sounds based on the letters of a user's name I created a virtual instrument using Python a

Paolo Sidera 2 Feb 11, 2022
The Best Multipurpose Discord Bot!

Polsu The Best Multipurpose Discord Bot! β€’ Introduction β€’ Screenshots β€’ Setup β€’ License Introduction Polsu is a Multipurpose Discord Bot. Polsu has a

Polsulpicien 1 Nov 09, 2021
VC-Music , Playing music without bot.

VC-Userbot A Telegram Userbot to play or streaming Audio and Video songs / files in Telegram Voice Chats. It's made with PyTgCalls and Pyrogram Requir

RioProjectX 8 Aug 04, 2022
An API wrapper for Discord written in Python.

disnake A modern, easy to use, feature-rich, and async ready API wrapper for Discord written in Python. About disnake All the contributors and develop

557 Jan 05, 2023
A python script to acquire multiple aws ec2 instances in a forensically sound-ish way

acquire_ec2.py The script acquire_ec2.py is used to automatically acquire AWS EC2 instances. The script needs to be run on an EC2 instance in the same

Deutsche Telekom Security GmbH 31 Sep 10, 2022
Telegram Client and Bot that use Artificial Intelligence to auto-reply to scammers and waste their time

scamminator Blocking a scammer is not enough. It is time to fight back. Wouldn't be great if there was a tool that uses Artificial Intelligence to rep

Federico Galatolo 6 Nov 12, 2022
A maintained fork of Danny's discord.py

Nextcord A modern, easy-to-use, feature-rich, and async-ready API wrapper for Discord written in Python. Fork notice This is a fork of discord.py, whi

977 Jan 05, 2023
A Python implementation of a discord bot that acts as a server scanner similar to Copenheimer.

Bad Copenheimer A Python impelentation of a discord bot that acts as a server scanner. This is a discord bot that will scan ip adresses to see if they

pilot1782 69 Dec 16, 2022
Dicha herramienta esta creada con una api... esta api permite enviar un SMS cada 12 horas dependiendo del pais... Hay algunos paises y operadoras no estΓ‘n soportados.

SMSFree pkg install python3 pip install requests git clone https://github.com/Hidden-parker/SMSFree cd SMSFree python sms.py DISFRUTA... Dicha herrami

piter 2 Nov 14, 2021
Backend for Indipe client

Betsushi Betsu (εˆ₯), the japanese word meaning "another" and Shiharai (支払い) meaning "payment". Hence the name Betsushi was derived. Introduction This i

Sudodevs 3 Feb 09, 2022
Melissa Songs is a telegram bot to easily find songs sending music snippets and search tracks πŸ’ƒπŸ½πŸŽ΅

Find songs on Telegram, like on Shazam... 😎 Open on telegram Β· Report Bug Β· Request Feature ⬇️ Installation To get a local copy installed and working

Joaquim Roque 21 Nov 10, 2022
A Twitter Bot that retweets and likes tweets with the hashtag #girlscriptwoc and #girlscript, and also follows the user.

GirlScript Winter of Contributing Twitter Bot A Twitter Bot that retweets and likes tweets with the hashtag #girlscriptwoc and #girlscript, and also f

Pranay Gupta 9 Dec 15, 2022
A bot for the [email protected] Discord server.

KittyBot - a sentient Discord bot! Key Notes An open-source, community-powered bot for the [email 

Ollie 11 Dec 06, 2022
DeFi wallet on Chia Network.

DeFi wallet on Chia Network.

GobyWallet 21 Aug 12, 2022
Instagram GiftShop Scam Killer

Instagram GiftShop Scam Killer A basic tool for Windows which kills acess to any giftshop scam from Instagram. Protect your Instagram account from the

1 Mar 31, 2022