Breaking the Curse of Space Explosion: Towards Effcient NAS with Curriculum Search
Pytorch implementation for "Breaking the Curse of Space Explosion: Towards Effcient NAS with Curriculum Search".
Requirements
python>=3.7, torch==1.5.0, torchvision==0.6.0, graphviz
Please install all the requirements in requirements.txt.
Datasets
We consider two benchmark classification datsets, including CIFAR-10 and ImageNet.
CIFAR-10 can be automatically downloaded by torchvision.
ImageNet needs to be manually downloaded (preferably to a SSD) following the instructions here.
Training Method
Curriculum search on CIFAR-10
python search.py -o outputs/search
Evaluation Method
- Put the searched architectures in
cnas/model/genotypes.pyas follows.
CNAS = Genotype.from_arch(
normal_arch=[('dil_conv_3x3', 1, 2),
('sep_conv_3x3', 0, 2),
('sep_conv_3x3', 0, 3),
('skip_connect', 1, 3),
('sep_conv_3x3', 1, 4),
('max_pool_3x3', 3, 4),
('sep_conv_3x3', 2, 5),
('sep_conv_3x3', 4, 5)],
normal_concat=[2, 3, 4, 5],
reduced_arch=[('sep_conv_3x3', 0, 2),
('skip_connect', 1, 2),
('dil_conv_5x5', 2, 3),
('skip_connect', 1, 3),
('dil_conv_3x3', 2, 4),
('sep_conv_3x3', 1, 4),
('sep_conv_5x5', 0, 5),
('sep_conv_3x3', 3, 5)],
reduced_concat=[2, 3, 4, 5])
- Evaluate the searched architecture on CIFAR-10 and ImageNet dataset using the following scripts.
Evaluation on CIFAR-10:
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch \
--nproc_per_node=2 --master_port=23333 \
eval_arch.py \
--arch CNAS --init_channels 36 --layers 20 \
-o outputs/cifar10
Evaluation on ImageNet:
CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch \
--nproc_per_node=4 --master_port=22333 \
eval_arch.py \
--max_epochs 250 --scheduler linear \
--dataset imagenet --data /path/to/imagenet \
--batch_size 64 --no_bias_decay --num_workers 8 \
--arch CNAS --init_channels 48 --layers 14 \
-o outputs/imagenet
Pretrained models
We have released our CNAS ImageNet pretrained model (top-1 accuracy 75.4%, top-5 accuracy 92.6%) on here.
You can use the following codes to load the ptrained models:
from cnas.model.eval import cnas_imagenet
model = cnas_imagenet(pretrained=True)
Citation
If you use any part of our code in your research, please cite our paper:
@InProceedings{guo2020breaking,
title = {Breaking the Curse of Space Explosion: Towards Efficient NAS with Curriculum Search},
author = {Guo, Yong and Chen, Yaofo and Zheng, Yin and Zhao, Peilin and Chen, Jian and Huang, Junzhou and Tan, Mingkui},
booktitle = {Proceedings of the 37th International Conference on Machine Learning},
year = {2020},
pages = {3822--3831}
}
3 Jan 7, 2022
79 Dec 19, 2022
21 Dec 13, 2022
14 Sep 30, 2022
232 Dec 25, 2022
57 Dec 15, 2022
21 Dec 27, 2022
41 Dec 12, 2022
772 Dec 31, 2022
1.4k Dec 25, 2022
1 Feb 07, 2022
2.2k Jan 07, 2023
1 Feb 07, 2022
1.3k Dec 26, 2022
1.4k Jan 01, 2023
3 Nov 06, 2022
59 Dec 09, 2022
110 Dec 20, 2022
74 Dec 07, 2022
42 Nov 10, 2022
47 Jun 30, 2022
4 Jul 20, 2022
1.1k Dec 16, 2022
995 Jan 08, 2023
44 Dec 17, 2022
227 Dec 26, 2022
117 Jan 03, 2023
4 Dec 29, 2021