CrossViT
This repository is the official implementation of CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification. ArXiv
If you use the codes and models from this repo, please cite our work. Thanks!
@inproceedings{
chen2021crossvit,
title={{CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification}},
author={Chun-Fu (Richard) Chen and Quanfu Fan and Rameswar Panda},
booktitle={International Conference on Computer Vision (ICCV)},
year={2021}
}
Installation
To install requirements:
pip install -r requirements.txt
With conda:
conda create -n crossvit python=3.8
conda activate crossvit
conda install pytorch=1.7.1 torchvision cudatoolkit=11.0 -c pytorch -c nvidia
pip install -r requirements.txt
Data preparation
Download and extract ImageNet train and val images from http://image-net.org/. The directory structure is the standard layout for the torchvision datasets.ImageFolder, and the training and validation data is expected to be in the train/ folder and val folder respectively:
/path/to/imagenet/
train/
class1/
img1.jpeg
class2/
img2.jpeg
val/
class1/
img3.jpeg
class/2
img4.jpeg
Pretrained models
We provide models trained on ImageNet1K. You can find models here. And you can load pretrained weights into models by add --pretrained flag.
Training
To train crossvit_9_dagger_224 on ImageNet on a single node with 8 gpus for 300 epochs run:
python -m torch.distributed.launch --nproc_per_node=8 --use_env main.py --model crossvit_9_dagger_224 --batch-size 256 --data-path /path/to/imagenet
Other model names can be found at models/crossvit.py.
Multinode training
Distributed training is available via Slurm and submitit:
To train a crossvit_9_dagger_224 model on ImageNet on 4 nodes with 8 gpus each for 300 epochs:
python run_with_submitit.py --nodes 4 --model crossvit_9_dagger_224 --data-path /path/to/imagenet --batch-size 128 --warmup-epochs 30
Or you can start process on each machine maunally. E.g. 2 nodes, each with 8 gpus.
Machine 0:
python -m torch.distributed.launch --nproc_per_node=8 --master_addr=MACHINE_0_IP --master_port=AVAILABLE_PORT --nnodes=2 --node_rank=0 main.py --model crossvit_9_dagger_224 --batch-size 256 --data-path /path/to/imagenet
Machine 1:
python -m torch.distributed.launch --nproc_per_node=8 --master_addr=MACHINE_0_IP --master_port=AVAILABLE_PORT --nnodes=2 --node_rank=1 main.py --model crossvit_9_dagger_224 --batch-size 256 --data-path /path/to/imagenet
Note that: some slurm configurations might need to be changed based on your cluster.
Evaluation
To evaluate a pretrained model on crossvit_9_dagger_224:
python -m torch.distributed.launch --nproc_per_node=8 --use_env main.py --model crossvit_9_dagger_224 --batch-size 128 --data-path /path/to/imagenet --eval --pretrained
1 Dec 24, 2021
129 Dec 11, 2022
1.4k Dec 30, 2022
41 Jan 2, 2023
52 Nov 22, 2022
274 Dec 28, 2022
109 Dec 28, 2022
6 Sep 21, 2022
26 Nov 22, 2022
2 Oct 02, 2022
315 Dec 26, 2022
41 Oct 21, 2022
3 Mar 06, 2022
314 Nov 25, 2022
117 Dec 29, 2022
1 Feb 12, 2022
46 Dec 20, 2022
90 Dec 13, 2022
56 Nov 01, 2022
298 Jan 07, 2023
2 Apr 23, 2022
47 Nov 06, 2022
1 Jan 12, 2022
2 Oct 18, 2021
72 Dec 17, 2022
19 Dec 29, 2022
9 May 10, 2022
42 Dec 06, 2022