Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
Discord Multitool made in python 3.9

XTool Discord Multitool 24 Features: Webhook Delete VC Lagger Fast Token Checker Mass Report [Not Done] Token rape 2K Characters Bypass Block bypass M

Tiie 50 Dec 20, 2022
This is a python wrapper for "the best api in the world"

This is a python wrapper for my api api_url = "https://api.dhravya.me/" This wrapper now has async support, its basically the same except it uses asyn

Dhravya Shah 3 Dec 21, 2021
A telegram bot help you to get stylish fonts and text

Stylish Font Bot 🐿 This is a telegram bot help you to get stylish fonts and text. Config Vars 🤖 API_HASH: Get this value from my.telegram.org. API_K

MSTL updates 1 Nov 08, 2021
Simple Telegram Bot for generating BalckPearl BBCode Templates

blackpearl-bbcode-bot Simple Telegram Bot for generating BlackPearl BBCode Templates Written in Pyrogram Features - 🎉 IMDB Info fetching from files -

D. Luffy 5 Oct 19, 2022
Opencontactbook - Bulk-manage large numbers of vCard contacts with built-in geolocation

Open Contact Book Open Contact Book is a buiness-oriented, cross-platform, Pytho

Aurélien PIERRE 2 Aug 08, 2022
Some examples regarding how to use the Twitter APIs for academic research

Twitter Developer Platform: Using Twitter APIs for Academic Research All the scripts require a config.ini file in which the keys are put. There is a t

Federico Bianchi 6 Feb 13, 2022
A group management bot written in python3 using the python-telegram-bot library.

Chika Fujiwara A modular telegram Python bot running on python3 with an sqlalchemy database. Originally a Marie fork, Chika was created for personal u

Wahyusaputra 3 Feb 12, 2022
This repository contains Python code examples using the MoneyMoov API

Python Examples This repository contains Python code examples using the MoneyMoov API. The examples are written to operate in the NoFrixion sandbox en

nofrixion 1 Feb 08, 2022
A Python wrapper for the tesseract-ocr API

tesserocr A simple, Pillow-friendly, wrapper around the tesseract-ocr API for Optical Character Recognition (OCR). tesserocr integrates directly with

Fayez 1.7k Jan 03, 2023
Python-random-quote - A file-based quote bot written in Python

Let's Write a Python Quote Bot! This repository will get you started with building a quote bot in Python. It's meant to be used along with the Learnin

amir mohammad fateh 1 Jan 02, 2022
Check your accounts/tokens fast with our checker!

Discord_Account_Checker How to use? Installing library's pip install -r reqs.txt Loading accounts Load your accounts to accounts.txt file. Launch pyth

1 Jan 11, 2022
A Telegram bot for combining emojis.

combimoji combimoji is a Telegram bot for combining emojis. How can I use it? You can find combimoji at @combimoji_bot, however it is not up (as of No

Yarema Mishchenko 2 Dec 02, 2021
Discord Token Generator based on HTTPX, makes unverified tokens and automatically joins your server! this is used for memberboosting

Discord Token Generator | 2021 Features: (1) hCaptcha Bypasser, latest hfuck.py Updated by me (2) Free Proxy Support/Scrapper (3) Custom Realistic Dat

2 Nov 30, 2021
A Telegram bot to send messages in Telegram groups or Channels using bots anonymously.

Group-chatting-bot A bot to send messeges to group using bot telegram bot ❤️ Support Made with Python3

Pyrogramers 16 Nov 06, 2022
With Google Drive API. My computer and my phone are in love now.

Channel trought Google Drive Google Drive API In this case, "Google Drive App" is the program. To install everything you need(has some extra things),

Luis Quiñones Requelme 1 Dec 15, 2021
A modern, easy to use, feature-rich, and async ready API wrapper for Discord written in Python.

A modern, easy to use, feature-rich, and async ready API wrapper for Discord written in Python. Key Features Modern Pythonic API using async and await

Senpai Development 4 Nov 05, 2021
Home Assistant custom integration for controlling Powered by Tuya (PBT) devices using Tuya Open API, officially maintained by the Tuya Developer Team.

Tuya Home Assistant Integration Home Assistant custom integration for controlling Powered by Tuya (PBT) devices using Tuya Open API, officially mainta

Tuya 704 Jan 03, 2023
Rust UserBot, Telegram istifadəsini asanlaşdıran bir proyektdir.

RUST USERBOT 🇦🇿 Rust UserBot, Telegram istifadəsini asanlaşdıran bir proyektdir. Qurulum Heroku Serverə qurulum git clone https://github.com/rustres

1 Oct 25, 2021
Simple progressbar for discord

⚙️ DiscordProgressbar 📂 Установка | Installation pip install discordbar 📚 Документация | Documentation 📞 Связаться со мной | Сontact with me 📜 Ли

DenyS 26 Nov 30, 2022
Python interface to the World Bank Indicators and Climate APIs

wbpy A Python interface to the World Bank Indicators and Climate APIs. Readthedocs Github source World Bank API docs The Indicators API lets you acces

Matt Duck 47 Oct 31, 2022