Skip to content

LSTC: Boosting Atomic Action Detection with Long-Short-Term Context

Notifications You must be signed in to change notification settings

TencentYoutuResearch/ActionDetection-LSTC

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

LSTC: Boosting Atomic Action Detection with Long-Short-Term Context

This Repository contains the code on AVA of our ACM MM 2021 paper: LSTC: Boosting Atomic Action Detection with Long-Short-Term Context

Installation

See INSTALL.md for details on installing the codebase, including requirement and environment settings

Data

For data preparation and setup, our LSTC strictly follows the processing of PySlowFast, See DATASET.md for details on preparing the data.

Run the code

We take SlowFast-ResNet50 as an example

  • train the model
python3 tools/run_net.py --cfg config/AVA/SLOWFAST_32x12_R50_LFB.yaml \
    AVA.FEATURE_BANK_PATH 'path/to/feature/bank/folder' \
    TRAIN.CHECKPOINT_FILE_PATH 'path/to/pretrained/backbone' \
    OUTPUT_DIR 'path/to/output/folder'
  • test the model
python3 tools/run_net.py --cfg config/AVA/SLOWFAST_32x12_R50_LFB.yaml \
    AVA.FEATURE_BANK_PATH 'path/to/feature/bank/folder' \
    OUTPUT_DIR 'path/to/output/folder' \
    TRAIN.ENABLE False \ 
    TEST.ENABLE True

If you want to start the DDP training from command line with torch.distributed.launch, please set start_method='cmd' in tools/run_net.py

Resource

The codebase provide following resources for fast training and validation

Pretrained backbone on Kinetics

backbone dataset model type link
ResNet50 Kinetics400 Caffe2 Google Drive/Baidu Disk (Code: y1wl)
ResNet101 Kinetics600 Caffe2 Google Drive/Baidu Disk (Code: slde)

Extracted long term feature bank

backbone feature bank (LMDB) dimension
ResNet50 Google Drive 1280
ResNet101 Google Drive 2304

Checkpoint file

backbone checkpoint model type
ResNet50 Google Drive/Baidu Disk (Code: fi0s) pytorch
ResNet101 Google Drive/Baidu Disk (Code: g63o) pytorch

Acknowledgement

This codebase is built upon PySlowFast.

Citation

If you find this repository helps your research, please refer following paper

@InProceedings{Yuxi_2021_ACM,
  author = {Li, Yuxi and Zhang, Boshen and Li, Jian and Wang, Yabiao and Wang, Chengjie and Li, Jilin and Huang, Feiyue and Lin, Weiyao},
  title = {LSTC: Boosting Atomic Action Detection with Long-Short-Term Context},
  booktitle = {ACM Conference on Multimedia},
  month = {October},
  year = {2021}
} 

About

LSTC: Boosting Atomic Action Detection with Long-Short-Term Context

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published