by Zhaowei Cai, and Nuno Vasconcelos.
This implementation is written by Zhaowei Cai at UC San Diego.
EdMIPS is an efficient algorithm to search the optimal mixed-precision neural network directly without proxy task on ImageNet given computation budgets. It can be applied to many popular network architectures, including ResNet, GoogLeNet, and Inception-V3. More details can be found in the paper.
If you use our code/model/data, please cite our paper:
@inproceedings{cai20edmips,
author = {Zhaowei Cai and Nuno Vasconcelos},
Title = {Rethinking Differentiable Search for Mixed-Precision Neural Networks},
booktitle = {CVPR},
Year = {2020}
}
-
Install PyTorch and ImageNet dataset following the official PyTorch ImageNet training code.
-
Clone the EdMIPS repository, and we'll call the directory that you cloned EdMIPS into
EdMIPS_ROOTgit clone https://github.com/zhaoweicai/EdMIPS.git cd EdMIPS_ROOT/
You can start training EdMIPS. Take ResNet-18 for example.
python search.py \
-a mixres18_w1234a234 --epochs 25 --step-epoch 10 --lr 0.1 --lra 0.01 --cd 0.00335 -j 16 \
[your imagenet-folder with train and val folders]
The other network architectures are also available, including ResNet-50, GoogLeNet and Inception-V3.
After the EdMIPS searching is finished, with the checkpoint arch_checkpoint.pth.tar, you can start to train the classification model with the learned bit allocation.
python main.py \
-a quantres18_cfg --epochs 95 --step-epoch 30 -j 16 \
--ac arch_checkpoint.pth.tar \
[your imagenet-folder with train and val folders]
The results are shown as following:
| network | precision | bit | --cd | top-1/5 acc. | model |
|---|---|---|---|---|---|
| ResNet-18 | uniform | 2.0 | 65.1/86.2 | download | |
| ResNet-18 | mixed | 1.992 | 0.00335 | 65.9/86.5 | download |
| ResNet-50 | uniform | 2.0 | 70.6/89.8 | download | |
| ResNet-50 | mixed | 2.007 | 0.00015 | 72.1/90.6 | download |
| GoogleNet | uniform | 2.0 | 64.8/86.3 | download | |
| GoogleNet | mixed | 1.994 | 0.00045 | 67.8/88.0 | download |
| Inception-V3 | uniform | 2.0 | 71.0/89.9 | download | |
| Inception-V3 | mixed | 1.982 | 0.0015 | 72.4/90.7 | download |
-
The training of EdMIPS has some variance. Tune
--cda little bit to get the optimal bit allocation you want. -
The BitOps are counted only on the quantized layers. They are normalized to the bit space as in the above table.
-
Since some changes have been made after the paper submission, you may get slightly worse performances (0.1~0.2 points) than those in the paper.
If you encounter any issue when using our code/model, please let me know.