Densely connected normalizing flows
This repository is the official implementation of NeurIPS 2021 paper Densely connected normalizing flows. Poster available here.
Setup
- CUDA 11.1
- Python 3.8
pip install -r requirements.txt
pip install -e .
Training
cd ./experiments/image
CIFAR-10:
python train.py --epochs 400 --batch_size 64 --optimizer adamax --lr 1e-3 --gamma 0.9975 --warmup 5000 --eval_every 1 --check_every 10 --dataset cifar10 --augmentation eta --block_conf 6 4 1 --layers_conf 5 6 20 --layer_mid_chnls 48 48 48 --growth_rate 10 --name DF_74_10
python train_more.py --model ./log/cifar10_8bit/densenet-flow/expdecay/DF_74_10 --new_lr 2e-5 --new_epochs 420
ImageNet32:
python train.py --epochs 20 --batch_size 64 --optimizer adamax --lr 1e-3 --gamma 0.95 --warmup 5000 --eval_every 1 --check_every 10 --dataset imagenet32 --augmentation eta --block_conf 6 4 1 --layers_conf 5 6 20 --layer_mid_chnls 48 48 48 --growth_rate 10 --name DF_74_10
python train_more.py --model ./log/imagenet32_8bit/densenet-flow/expdecay/DF_74_10 --new_lr 2e-5 --new_epochs 22
ImageNet64:
python train.py --epochs 10 --batch_size 32 --optimizer adamax --lr 1e-3 --gamma 0.95 --warmup 5000 --eval_every 1 --check_every 10 --dataset imagenet64 --augmentation eta --block_conf 6 4 1 --layers_conf 5 6 20 --layer_mid_chnls 48 48 48 --growth_rate 10 --name DF_74_10
python train_more.py --model ./log/imagenet64_8bit/densenet-flow/expdecay/DF_74_10 --new_lr 2e-5 --new_epochs 11
CelebA:
python train.py --epochs 50 --batch_size 32 --optimizer adamax --lr 1e-3 --gamma 0.95 --warmup 5000 --eval_every 1 --check_every 10 --dataset celeba --augmentation horizontal_flip --block_conf 6 4 1 --layers_conf 5 6 20 --layer_mid_chnls 48 48 48 --growth_rate 10 --name DF_74_10
python train_more.py --model ./log/celeba_8bit/densenet-flow/expdecay/DF_74_10 --new_lr 2e-5 --new_epochs 55
Note: Download instructions for ImageNet and CelebA are defined in denseflow/data/datasets/image/{dataset}.py
Evaluation
CIFAR-10:
python eval_loglik.py --model PATH_TO_MODEL --k 1000 --kbs 50
ImageNet32:
python eval_loglik.py --model PATH_TO_MODEL --k 200 --kbs 50
ImageNet64 and CelebA:
python eval_loglik.py --model PATH_TO_MODEL --k 200 --kbs 25
Model weights
Model weights are stored here.
Samples generation
Generated samples are stored in PATH_TO_MODEL/samples
python eval_sample.py --model PATH_TO_MODEL
Note: PATH_TO_MODEL has to contain check directory.
ImageNet 32x32
ImageNet 64x64
CelebA
Acknowledgements
Significant part of this code benefited from SurVAE [1] code implementation, available under MIT license.
References
[1] Didrik Nielsen, Priyank Jaini, Emiel Hoogeboom, Ole Winther, and Max Welling. Survae flows: Surjections to bridge the gap between vaes and flows. InAdvances in Neural Information Processing Systems 33. Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020.
12 Jan 10, 2022
87 Dec 21, 2022
217 Jan 01, 2023
7 Oct 14, 2022
577 Dec 17, 2022
3.3k Jan 06, 2023
57 Nov 14, 2022
70 Dec 04, 2022
59 Dec 14, 2022
28 Dec 12, 2022
6 Dec 05, 2022
15 Dec 22, 2022
9 Nov 18, 2022
46 Dec 21, 2022
38 Dec 13, 2022
40 Oct 07, 2022
406 Dec 23, 2022
23 Dec 22, 2022
4.9k Dec 31, 2022
21 Jan 02, 2022
