Breaking the Curse of Space Explosion: Towards Effcient NAS with Curriculum Search
Pytorch implementation for "Breaking the Curse of Space Explosion: Towards Effcient NAS with Curriculum Search".
Requirements
python>=3.7, torch==1.5.0, torchvision==0.6.0, graphviz
Please install all the requirements in requirements.txt.
Datasets
We consider two benchmark classification datsets, including CIFAR-10 and ImageNet.
CIFAR-10 can be automatically downloaded by torchvision.
ImageNet needs to be manually downloaded (preferably to a SSD) following the instructions here.
Training Method
Curriculum search on CIFAR-10
python search.py -o outputs/search
Evaluation Method
- Put the searched architectures in
cnas/model/genotypes.pyas follows.
CNAS = Genotype.from_arch(
normal_arch=[('dil_conv_3x3', 1, 2),
('sep_conv_3x3', 0, 2),
('sep_conv_3x3', 0, 3),
('skip_connect', 1, 3),
('sep_conv_3x3', 1, 4),
('max_pool_3x3', 3, 4),
('sep_conv_3x3', 2, 5),
('sep_conv_3x3', 4, 5)],
normal_concat=[2, 3, 4, 5],
reduced_arch=[('sep_conv_3x3', 0, 2),
('skip_connect', 1, 2),
('dil_conv_5x5', 2, 3),
('skip_connect', 1, 3),
('dil_conv_3x3', 2, 4),
('sep_conv_3x3', 1, 4),
('sep_conv_5x5', 0, 5),
('sep_conv_3x3', 3, 5)],
reduced_concat=[2, 3, 4, 5])
- Evaluate the searched architecture on CIFAR-10 and ImageNet dataset using the following scripts.
Evaluation on CIFAR-10:
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch \
--nproc_per_node=2 --master_port=23333 \
eval_arch.py \
--arch CNAS --init_channels 36 --layers 20 \
-o outputs/cifar10
Evaluation on ImageNet:
CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch \
--nproc_per_node=4 --master_port=22333 \
eval_arch.py \
--max_epochs 250 --scheduler linear \
--dataset imagenet --data /path/to/imagenet \
--batch_size 64 --no_bias_decay --num_workers 8 \
--arch CNAS --init_channels 48 --layers 14 \
-o outputs/imagenet
Pretrained models
We have released our CNAS ImageNet pretrained model (top-1 accuracy 75.4%, top-5 accuracy 92.6%) on here.
You can use the following codes to load the ptrained models:
from cnas.model.eval import cnas_imagenet
model = cnas_imagenet(pretrained=True)
Citation
If you use any part of our code in your research, please cite our paper:
@InProceedings{guo2020breaking,
title = {Breaking the Curse of Space Explosion: Towards Efficient NAS with Curriculum Search},
author = {Guo, Yong and Chen, Yaofo and Zheng, Yin and Zhao, Peilin and Chen, Jian and Huang, Junzhou and Tan, Mingkui},
booktitle = {Proceedings of the 37th International Conference on Machine Learning},
year = {2020},
pages = {3822--3831}
}
3 Jan 7, 2022
79 Dec 19, 2022
21 Dec 13, 2022
14 Sep 30, 2022
232 Dec 25, 2022
57 Dec 15, 2022
21 Dec 27, 2022
41 Dec 12, 2022
182 Jan 07, 2023
9 Apr 29, 2022
0 Feb 04, 2022
28 Nov 18, 2022
164 Dec 30, 2022
592 Jan 03, 2023
55 Jan 04, 2023
157 Dec 23, 2022
6.5k Jan 01, 2023
9 Dec 11, 2022
21 Jul 27, 2022
30 Jan 06, 2023
2.3k Dec 17, 2022
181 Dec 19, 2022
37 Nov 09, 2022
28 Dec 09, 2022
7 May 25, 2022
24 Nov 03, 2022
216 Dec 26, 2022
0 Dec 13, 2021