Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
this synchronizes my appearances with my calendar

Josh's Schedule Synchronizer Here's the "problem:" I use a spreadsheet to maintain all my public appearances. I check the spreadsheet as often as poss

Josh Long 2 Oct 18, 2021
A userbot made for telegram

𝚃𝙷𝙴 𝙼𝙰𝙵𝙸𝙰𝙱𝙾𝚃 This is a userbot made for telegram. I made this userbot with help of all other userbots available in telegram. All credits go

MafiaBotOP 8 Apr 08, 2022
Python wrapper to simplify calls to AncestryDNA API.

AncestryDNA API wrapper Ancestry exposes an undocumented REST API for its DNA features. This Python wrapper inventories the available calls, and expos

Matt 2 Jun 10, 2022
Offline reverse geocoder in Python using sqlite3

rgeocode Offline reverse geocoder rgeocode accepts a geographic coordinate pair (latitude and longitude) and returns a list containing the name of: A

Venkat 7 Dec 01, 2021
OpenAPI Generator allows generation of API client libraries (SDK generation), server stubs, documentation and configuration automatically given an OpenAPI Spec (v2, v3)

OpenAPI Generator Master (5.3.1): 5.4.x (5.4.x): 6.0.x (6.0.x): ⭐ ⭐ ⭐ If you would like to contribute, please refer to guidelines and a list of open t

OpenAPI Tools 14.8k Jan 04, 2023
A youtube search telegram bot.

YouTube-Search-Bot A youtube search telegram bot. Made with Python3 (C) @FayasNoushad Copyright permission under MIT License License - https://github

Fayas Noushad 22 Nov 12, 2022
The records of 42 million users from a third-party version of the popular Telegram messaging app have just been Iranian accounts leaked

TelegramDatabase About The records of 42 million users from a third-party version of the popular Telegram messaging app have just been Iranian account

Hamed Mohammadvand 10 Jan 14, 2022
Wechat-file-cleaner - Clean files in PC WeChat FileStorage directory

Wechat-file-cleaner - Clean files in PC WeChat FileStorage directory

Xingjian Zhang 1 Feb 06, 2022
Google Sheets Python API v4

pygsheets - Google Spreadsheets Python API v4 A simple, intuitive library for google sheets which gets your work done. Features: Open, create, delete

Nithin Murali 1.4k Jan 08, 2023
Telegram chats reader with python

Telegram chat reader Программа полностью сливает чаты телеграм в базу данных PostgreSQL. Для использования нужен развернутый сервер постгрес и телегра

Anton 4 Dec 24, 2021
Monitoring plugin for MikroTik devices

check_routeros - Monitoring MikroTik devices This is a monitoring plugin for Icinga, Nagios and other compatible monitoring solutions to check MikroTi

DinoTools 6 Dec 24, 2022
gnosis safe tx builder

Ape Safe: Gnosis Safe tx builder Ape Safe allows you to iteratively build complex multi-step Gnosis Safe transactions and safely preview their side ef

228 Dec 22, 2022
Tiktok BOT 1 link Vidio otomatis : - Nambah view - Nambah love - Nambah share - Nambah love komen (semua komen ke love)

Install Bahannya pip install selenium pip install pyfiglet==0.7.5 Cara Install dan Menjalankannya Git-clone this repo! Install selenium and pyfiglet u

Ahmad Ghozali 1 Mar 01, 2022
Crystal Orb is a discord bot made from discord.py and python

Crystal orb Replacing barbot Overview Crystal Orb is a discord bot made from discord.py and python, Crystal Orb is for anti alt detection and other st

AlexyDaCoder 3 Nov 28, 2021
A customizable, multilanguage Telegram shop bot with Telegram Payments support

Greed A customizable, multilanguage Telegram shop bot with Telegram Payments support! Demo Send a message to @greedtestbot on Telegram to view a demo

Stefano Pigozzi 328 Dec 29, 2022
Smilecreator4 - This site is for people who want to hack or want to learn it!

smilecreator4 This site is for people who want to hack or want to learn it! Furthermore, this program does not work without turning off Antivirus or W

1 Jan 04, 2022
Red-mail - Advanced email sending library for Python

Red Mail Next generation email sender What is it? Red Mail is an advanced email

Mikael Koli 313 Jan 08, 2023
A Telegram bot to download from Youtube server.

IDN-YoutubeDL-Bot A Telegram bot to download from Youtube server. Configs 📖 API_ID - Your APP ID. Get it from my.telegram.org API_HASH - Your API_HAS

IDNCoderX 4 Dec 02, 2022
A python library built on the API of the coderHub.sa, which helps you to fetch the challenges and more

coderHub A python library built on the API of the coderHub.sa, which helps you to fetch the challenges and more Installation • Features • Usage • Lice

TheAwiteb 5 Nov 04, 2022
A simple and easy to use musicbot in python and it uses lavalink.

Lavalink-MusicBot A simple and easy to use musicbot in python and it uses lavalink. ✨ Features plays music in your discord server well thats it i gues

Afnan 1 Nov 29, 2021