STAR

Official implementation of Sparse Transformer-based Action Recognition

Dataset

download NTU RGB+D 60 action recognition of 2D/3D skeleton from http://rose1.ntu.edu.sg/datasets/actionRecognition.asp

or use google drive

NTU60 NTU120

uzip data as the following file structure: $(project_folder)/raw/.\*skeleton or $(project_folder)/dataset/raw/.\*skeleton (create "raw" folder under $(project_folder) or $(project_folder)/dataset then put raw skeleton files under "raw" folder)

run the code below to generate dataset:

python datagen.py

Training

git fetch and checkout to "distributed" branch

python train_dist.py -#distributed training

Configuration

parser.set_defaults(gpu=True,
                    batch_size=128,
                    dataset_name='NTU',
                    dataset_root=osp.join(os.getcwd()),  # or dataset_root=osp.join(os.getcwd(), 'dataset')
                    load_model=False,
                    in_channels=9,
                    num_enc_layers=5,
                    num_conv_layers=2,
                    weight_decay=4e-5,
                    drop_rate=[0.4, 0.4, 0.4, 0.4],  # linear_attention, sparse_attention, add_norm, ffn
                    hid_channels=64,
                    out_channels=64,
                    heads=8,
                    data_parallel=False,
                    cross_k=5,
                    mlp_head_hidden=128)

parser.set_defaults(gpu=True,
                    batch_size=128,
                    dataset_name='NTU',
                    dataset_root=osp.join(os.getcwd()),
                    load_model=False,
                    in_channels=9,
                    num_enc_layers=5,
                    num_conv_layers=2,
                    weight_decay=4e-5,
                    drop_rate=[0.4, 0.4, 0.4, 0.4],  # linear_attention, sparse_attention, add_norm, ffn
                    hid_channels=128,
                    out_channels=128,
                    heads=8,
                    data_parallel=False,
                    cross_k=5,
                    mlp_head_hidden=128)

Official implementation of Sparse Transformer-based Action Recognition

Related tags

Overview

STAR

Dataset

Training

Configuration

Owner

Chonghan_Lee

NaturalCC is a sequence modeling toolkit that allows researchers and developers to train custom models

Active and Sample-Efficient Model Evaluation

Implementation for Curriculum DeepSDF

DeepLearning Anomalies Detection with Bluetooth Sensor Data

EXplainable Artificial Intelligence (XAI)

Supercharging Imbalanced Data Learning WithCausal Representation Transfer

A collection of resources, problems, explanations and concepts that are/were important during my Data Science journey

Source code for TACL paper "KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation".

PyTorch implementation of the REMIND method from our ECCV-2020 paper "REMIND Your Neural Network to Prevent Catastrophic Forgetting"

TEA: A Sequential Recommendation Framework via Temporally Evolving Aggregations

Exploration-Exploitation Dilemma Solving Methods

AdaMML: Adaptive Multi-Modal Learning for Efficient Video Recognition

2021-MICCAI-Progressively Normalized Self-Attention Network for Video Polyp Segmentation

Static Features Classifier - A static features classifier for Point-Could clusters using an Attention-RNN model

Open source hardware and software platform to build a small scale self driving car.

Numbering permanent and deciduous teeth via deep instance segmentation in panoramic X-rays

Multi-modal Content Creation Model Training Infrastructure including the FACT model (AI Choreographer) implementation.

[MedIA2021]MIDeepSeg: Minimally Interactive Segmentation of Unseen Objects from Medical Images Using Deep Learning

A pytorch implementation of MBNET: MOS PREDICTION FOR SYNTHESIZED SPEECH WITH MEAN-BIAS NETWORK

A python/pytorch utility library