STAM - Pytorch
Implementation of STAM (Space Time Attention Model), yet another pure and simple SOTA attention model that bests all previous models in video classification. This corroborates the finding of TimeSformer. Attention is all we need.
Install
$ pip install stam-pytorch
Usage
import torch
from stam_pytorch import STAM
model = STAM(
dim = 512,
image_size = 256, # size of image
patch_size = 32, # patch size
num_frames = 5, # number of image frames, selected out of video
space_depth = 12, # depth of vision transformer
space_heads = 8, # heads of vision transformer
space_mlp_dim = 2048, # feedforward hidden dimension of vision transformer
time_depth = 6, # depth of time transformer (in paper, it was shallower, 6)
time_heads = 8, # heads of time transformer
time_mlp_dim = 2048, # feedforward hidden dimension of time transformer
num_classes = 100, # number of output classes
space_dim_head = 64, # space transformer head dimension
time_dim_head = 64, # time transformer head dimension
dropout = 0., # dropout
emb_dropout = 0. # embedding dropout
)
frames = torch.randn(2, 5, 3, 256, 256) # (batch x frames x channels x height x width)
pred = model(frames) # (2, 100)
Citations
@misc{sharir2021image,
title = {An Image is Worth 16x16 Words, What is a Video Worth?},
author = {Gilad Sharir and Asaf Noy and Lihi Zelnik-Manor},
year = {2021},
eprint = {2103.13915},
archivePrefix = {arXiv},
primaryClass = {cs.CV}
}
191 Dec 31, 2022
2 Jun 5, 2022
58 Dec 21, 2022
159 Dec 22, 2022
54 Nov 21, 2022
8 Oct 29, 2022
272 Dec 23, 2022
1.4k Jan 1, 2023
2.9k Dec 31, 2022
2.5k Dec 30, 2022
81 Jun 30, 2022
166 Nov 27, 2022
191 Dec 28, 2022
7 Jul 21, 2022
189 Dec 07, 2022
31 Dec 05, 2022
118 Dec 06, 2022
35 Dec 08, 2022
109 Dec 23, 2022
18 Nov 05, 2022
2 Jan 14, 2022
4 Sep 29, 2022
85 Dec 03, 2022
136 Dec 28, 2022
520 Jan 04, 2023
295 Dec 26, 2022
3 Feb 22, 2022
28 Sep 16, 2022