Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
You cant check for conflicts until course enrolment actually opens. I wanted to do it earlier.

AcornICS I noticed that Acorn it does not let you check if a timetable is valid based on the enrollment cart, it also does not let you visualize it ea

Isidor Kaplan 2 Sep 16, 2021
Robocord is a bot created for the Pycord community.

Robocord is a bot created for the community of the Pycord Server. Just a bot created for Pycord Server. You can start pull requests, I will check it and if its good I will add it to the bot. 👍

Bruce 7 Jun 26, 2022
Manage Proxmox KVM Virtual Machines via Slack bot.

proxmox-slack-bot Create KVM Virtual Machines on Proxmox, the easy way. Not much works works here yet... Setup dev environment Setup fully editable st

Plenus Pyramis 3 Mar 20, 2022
This checks that your credit card is valid or not

Credit_card_Validator This checks that your credit card is valid or not. Where is the app ? main.exe is the application to run and main.py is the file

Ritik Ranjan 1 Dec 21, 2021
Create CDK projects with projen

The Projenator: I'll be back! Description This is a CDKv2 project that takes the grind out of setting up new cdk projects/implementations by using aut

Andrew 2 Dec 11, 2021
Discord Webhook Spammer (fastest)

Discord Webhook Spammer A simple fast asynchronous webhook spammer. Spammer Features Fast message spamming. Controllable speed. Noob friendly. Usage N

Varient 2 Apr 22, 2022
A simple Python script using Telethon to log all (or some) messages a user or bot account can see on Telegram.

telegram-logger A simple Python script using Telethon to log all (or some) messages a user or bot account can see on Telegram. Requirements Python 3.6

Richard 13 Oct 06, 2022
A quick way to verify your Climate Hack.AI (2022) submission locally!

Climate Hack.AI (2022) Submission Validator This repository contains code that allows you to quickly validate your Climate Hack.AI (2022) submission l

Jeremy 3 Mar 03, 2022
Step by Step Guide To Install Discord Py Master Branch on Replit

Guide to Install Discord Py Master Branch on Replit Step 1 Create an empty repl on replit Step 2 Add this Basic Code to the file main.py so as to chec

Pranav Saxena 7 Nov 18, 2022
quote is a python wrapper for the Goodreads Quote API, powered by gazpacho.

About quote is a python wrapper for the Goodreads Quote API, powered by gazpacho.

Max Humber 11 Nov 10, 2022
Zero2 Discord bot is written with Discord.py using Python.

Zero2 Discord bot is written with Discord.py using Python.

Siva Avanish 4 Nov 08, 2021
Deepak Clouds Torrent is a multipurpose Telegram Bot writen in Python for mirroring files on the Internet to our beloved Google Drive.

Deepak Clouds Torrent is a multipurpose Telegram Bot writen in Python for mirroring files on the Internet to our beloved Google Drive.

Deepak Clouds 37 Oct 28, 2022
The accompanying code for the paper "GMAT: Global Memory Augmentation for Transformers" (Ankit Gupta and Jonathan Berant).

GMAT: Global Memory Augmentation for Transformers This repository contains the accompanying code for the paper: "GMAT: Global Memory Augmentation for

Ankit Gupta 7 Oct 21, 2021
Seth Userbot with python

SETH-USERBOT DEPLOY TO HEROKU Group Support: String Session : Stay Support 🚀 ❁ LonamiWebs and Telethon © Credits ⚡ THANK YOU VERY MUCH FOR zeinzo Zei

seth 4 Jan 10, 2022
The first open-source PyTgCalls-based project.

SU Music Player — The first open-source PyTgCalls based Pyrogram bot to play music in voice chats Requirements FFmpeg NodeJS 15+ Python 3.7+ Deploymen

Calls Music 74 Nov 19, 2022
Discord Bot Sending Members - Leaked by BambiKu ( Me )

Wokify Bot Discord Bot Sending Members - Leaked by BambiKu ( Me ) Info The Bot was orginaly made by someone else! Ghost-Dev just wanted to sell "priva

bambiku 6 Jul 05, 2022
A simple telegram voting bot based on the python-telegram-bot api.

A simple telegram voting bot based on the python-telegram-bot api. *To make it more easy to use, I might make a C++ code in the future so you don't ha

3 Sep 13, 2021
Injector/automatic translator (using deepL API) for Tsukihime Remake

deepLuna Extractor/Editor/Translator/Injector for Tsukihime Remake About deepLuna, from "deepL", the machine translation service, and "Luna", the name

30 Dec 15, 2022
Weather Tracker, made with Python using Open Weather API

Weather Tracker Weather Tracker, made with Python using Open Weather API

Sahil Kumar 1 Feb 07, 2022
Want to play What Would Rather on your Server? Invite the bot now! 😏

What is this Bot? 👀 What You Would Rather? is a Guessing game where you guess one thing. Long Description short Take this example: You typed r!rather

FSP Gang s' YT 3 Oct 18, 2021