Code for paper: Towards Tokenized Human Dynamics Representation

Last update: May 31, 2022

Overview

Video Tokneization

Codebase for video tokenization, based on our paper Towards Tokenized Human Dynamics Representation.

Prerequisites (tested under Python 3.8 and CUDA 11.1)

apt-get install ffmpeg  
pip install torch==1.8  
pip install torchvision  
pip install pytorch-lightning  
pip install pytorch-lightning-bolts  
pip install aniposelib wandb gym test-tube ffmpeg-python matplotlib easydict scikit-learn

Data Preparation

Make a directory besides this repo and name it aistplusplus
Download from AIST++ website until it looks like

├── annotations
│   ├── cameras
│   ├── ignore_list.txt
│   ├── keypoints2d
│   ├── keypoints3d
│   ├── motions
│   └── splits
└── video_list.txt

How to run

Write one configuration file, e.g., configs/tan.yaml.
Run python pretrain.py --cfg configs/tan.yaml with GPU, which will create a folder under logs for this run. Folder name specified by the NAME in configuration file. Then run python cluster.py --cfg configs/tan.yaml (CPU-only) and check results in demo.ipynb.
Or you can download and unzip my training result into logs folder from here.

Code for paper: Towards Tokenized Human Dynamics Representation

Related tags

Overview

Video Tokneization

Prerequisites (tested under Python 3.8 and CUDA 11.1)

Data Preparation

How to run

Owner

Kenneth Li

PyTorch code for the NAACL 2021 paper "Improving Generation and Evaluation of Visual Stories via Semantic Consistency"

Anchor-free Oriented Proposal Generator for Object Detection

FLSim a flexible, standalone library written in PyTorch that simulates FL settings with a minimal, easy-to-use API

LQM - Improving Object Detection by Estimating Bounding Box Quality Accurately

SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data

[CVPR 2022 Oral] Crafting Better Contrastive Views for Siamese Representation Learning

HINet: Half Instance Normalization Network for Image Restoration

A Comparative Framework for Multimodal Recommender Systems

RuDOLPH: One Hyper-Modal Transformer can be creative as DALL-E and smart as CLIP

Implementation of SETR model, Original paper: Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers.

Code for the paper "How Attentive are Graph Attention Networks?"

Designing a Practical Degradation Model for Deep Blind Image Super-Resolution (ICCV, 2021) (PyTorch) - We released the training code!

Machine Learning Privacy Meter: A tool to quantify the privacy risks of machine learning models with respect to inference attacks, notably membership inference attacks

Trustworthy AI related projects

Official Pytorch implementation for AAAI2021 paper (RSPNet: Relative Speed Perception for Unsupervised Video Representation Learning)

Pytorch Implementation of "Desigining Network Design Spaces", Radosavovic et al. CVPR 2020.

Generate text captions for images from their CLIP embeddings. Includes PyTorch model code and example training script.

An efficient PyTorch implementation of the evaluation metrics in recommender systems.

Official implementation of VQ-Diffusion

NALSM: Neuron-Astrocyte Liquid State Machine