STAR

Official implementation of Sparse Transformer-based Action Recognition

Dataset

download NTU RGB+D 60 action recognition of 2D/3D skeleton from http://rose1.ntu.edu.sg/datasets/actionRecognition.asp

or use google drive

NTU60 NTU120

uzip data as the following file structure: $(project_folder)/raw/.\*skeleton or $(project_folder)/dataset/raw/.\*skeleton (create "raw" folder under $(project_folder) or $(project_folder)/dataset then put raw skeleton files under "raw" folder)

run the code below to generate dataset:

python datagen.py

Training

git fetch and checkout to "distributed" branch

python train_dist.py -#distributed training

Configuration

parser.set_defaults(gpu=True,
                    batch_size=128,
                    dataset_name='NTU',
                    dataset_root=osp.join(os.getcwd()),  # or dataset_root=osp.join(os.getcwd(), 'dataset')
                    load_model=False,
                    in_channels=9,
                    num_enc_layers=5,
                    num_conv_layers=2,
                    weight_decay=4e-5,
                    drop_rate=[0.4, 0.4, 0.4, 0.4],  # linear_attention, sparse_attention, add_norm, ffn
                    hid_channels=64,
                    out_channels=64,
                    heads=8,
                    data_parallel=False,
                    cross_k=5,
                    mlp_head_hidden=128)

parser.set_defaults(gpu=True,
                    batch_size=128,
                    dataset_name='NTU',
                    dataset_root=osp.join(os.getcwd()),
                    load_model=False,
                    in_channels=9,
                    num_enc_layers=5,
                    num_conv_layers=2,
                    weight_decay=4e-5,
                    drop_rate=[0.4, 0.4, 0.4, 0.4],  # linear_attention, sparse_attention, add_norm, ffn
                    hid_channels=128,
                    out_channels=128,
                    heads=8,
                    data_parallel=False,
                    cross_k=5,
                    mlp_head_hidden=128)

Official implementation of Sparse Transformer-based Action Recognition

Related tags

Overview

STAR

Dataset

Training

Configuration

Owner

Chonghan_Lee

[CVPR 2021 Oral] Variational Relational Point Completion Network

Align and Prompt: Video-and-Language Pre-training with Entity Prompts

Code release of paper Improving neural implicit surfaces geometry with patch warping

Vision transformers (ViTs) have found only limited practical use in processing images

[ECCVW2020] Robust Long-Term Object Tracking via Improved Discriminative Model Prediction (RLT-DiMP)

Lepard: Learning Partial point cloud matching in Rigid and Deformable scenes

Auto White-Balance Correction for Mixed-Illuminant Scenes

Assginment for UofT CSC420: Intro to Image Understanding

This is the code for CVPR 2021 oral paper: Jigsaw Clustering for Unsupervised Visual Representation Learning

Collection of machine learning related notebooks to share.

Point-NeRF: Point-based Neural Radiance Fields

Modeling CNN layers activity with Gaussian mixture model

DAFNe: A One-Stage Anchor-Free Deep Model for Oriented Object Detection

[Preprint] "Chasing Sparsity in Vision Transformers: An End-to-End Exploration" by Tianlong Chen, Yu Cheng, Zhe Gan, Lu Yuan, Lei Zhang, Zhangyang Wang

Automated Attendance Project Using Face Recognition

IDM: An Intermediate Domain Module for Domain Adaptive Person Re-ID,

RE3: State Entropy Maximization with Random Encoders for Efficient Exploration

Using this you can control your PC/Laptop volume by Hand Gestures (pinch-in, pinch-out) created with Python.

Gym environment for FLIPIT: The Game of "Stealthy Takeover"

Neural network pruning for finding a sparse computational model for controlling a biological motor task.