Human Action Controller - A human action controller running on different platforms.

Last update: Jul 20, 2022

Related tags

Overview

Human Action Controller (HAC)

Goal

A human action controller running on different platforms.

Fun	Easy-to-use
Accurate	Anywhere

Fun Examples

Mouse Control

Keyboard Control

Playing Game

Enhancing interaction

Solutions provided by HAC

Platform	Module	Progress
PC / Win10	Mouse Control	V
PC / Win10	Keyboard Control	V
PC / Ubuntu	Mouse Control
PC / Ubuntu	Keyboard Control

Getting started

Installation

$ pip install pyhac

Run the demo of mouse control

$ git clone https://github.com/dabit-lucas/hac.git
$ cd hac
$ python demo.py

Recording custom actions

$ python recording.py -d {action name} -k True

Press key "r" to start recording, the data will be saved into ./data

Training a custom module

Here is an example of a config file of action set,

{
    "actions": [
        "r_five",
        "r_zero",
        "l_five",
        "l_zero",
        "two_index_fingers_up",
        "two_index_fingers_down",
        "33",
        "55",
        "sit"
    ],
    "type": "gesture_only"
}

These actions form a model by running a training process:

$ python train.py --conf {path_of_action} --model_name {name_of_model}

The generated model will become a module. Take mouse control module as an exmaple, it can create mappings among actions and controls by the following code:

mouse_module = hac.add_module("mouse_control")
hac.set_init_module(mouse_module)

# create mapping between controls and actions
mouse_module.add_mouse_mapping("mouse_left_down", ["r_five", "r_zero"])
mouse_module.add_mouse_mapping("mouse_left_up", "r_five")
mouse_module.add_mouse_mapping("mouse_right_down", ["l_five", "l_zero"])
mouse_module.add_mouse_mapping("mouse_right_up", "l_five")
mouse_module.add_mouse_mapping("right_move_diff", ["r_five", "r_five"])
mouse_module.add_mouse_mapping("right_move_diff", ["r_zero", "r_zero"])
mouse_module.add_mouse_mapping("left_move_diff", ["l_five", "l_five"])
mouse_module.add_mouse_mapping("left_move_diff", ["l_zero", "l_zero"])
mouse_module.add_mouse_mapping("roll_up", "two_index_fingers_up")
mouse_module.add_mouse_mapping("roll_down", "two_index_fingers_down")

If the five gesture with a right hand shows in consecutive two frames ["r_five", "r_five"], then do control right_move_diff, which means moving the mouse cursor. The above description can be represented by the following code:

mouse_module.add_mouse_mapping("right_move_diff", ["r_five", "r_five"])

Development guideline

The structure of HAC

Community

Welcome to ask any question in issues.

Contributing

Any contribution is welcomed. Please fork this repo and summit a pull request.

Human Action Controller - A human action controller running on different platforms.

Related tags

Overview

Human Action Controller (HAC)

Goal

Fun Examples

Mouse Control

Keyboard Control

Playing Game

Enhancing interaction

Solutions provided by HAC

Getting started

Installation

Run the demo of mouse control

Recording custom actions

Training a custom module

Development guideline

Community

Contributing

Owner

Detectron2 is FAIR's next-generation platform for object detection and segmentation.

PICARD - Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models

Official tensorflow implementation for CVPR2020 paper “Learning to Cartoonize Using White-box Cartoon Representations”

An official source code for "Augmentation-Free Self-Supervised Learning on Graphs"

PyTorch version of Stable Baselines, reliable implementations of reinforcement learning algorithms.

Repository of 3D Object Detection with Pointformer (CVPR2021)

Training DiffWave using variational method from Variational Diffusion Models.

A highly modular PyTorch framework with a focus on Neural Architecture Search (NAS).

Playable Video Generation

Official repository of the paper Privacy-friendly Synthetic Data for the Development of Face Morphing Attack Detectors

Tools for robust generative diffeomorphic slice to volume reconstruction

This is an unofficial implementation of the paper “Student-Teacher Feature Pyramid Matching for Unsupervised Anomaly Detection”.

A motion tracking system for any arbitaray points in a video frame.

(ICCV 2021) PyTorch implementation of Paper "Progressive Correspondence Pruning by Consensus Learning"

CLIP + VQGAN / PixelDraw

The official PyTorch code implementation of "Personalized Trajectory Prediction via Distribution Discrimination" in ICCV 2021.

Official re-implementation of the Calibrated Adversarial Refinement model described in the paper Calibrated Adversarial Refinement for Stochastic Semantic Segmentation

The project was to detect traffic signs, based on the Megengine framework.

Official repository of the AAAI'2022 paper "Contrast and Generation Make BART a Good Dialogue Emotion Recognizer"

This repo tries to recognize faces in the dataset you created