Hi Rianne, This is a great code, and I have a little question about logp(zk), we hope p(zk) in VAE can be a distribution whose form is no fixed, but it seems that the calculate of logp(zk) in line81 of loss.py imply that p(zk) is a standard Gaussion. Are there some mistakes about my understanding?
Thank your for this code

hello Rianne: Thanks very much. I am a bit confused with line 44 in loss.py : loss = bce + beta * kl. Based on equation 3 in Tomczak's paper (Improving Variational Auto-Encoder Using Householder Flows), shouldn't "loss = bce - beta * kl "? Also, why use -ELBO instead of ELBO when reporting your metrics? Thanks

Hi Rianne,

This PR contains a 'minimal' set of changes to run the code with the latest PyTorch versions and Python 3 ( #1 #2 )

It is 'minimal' in the sense that I only made changes that affect functionality. There are additional cosmetic changes that could be made; e.g. Variable(), the volatile flag, and F.sigmoid() have been deprecated but they should not affect functionality.

I tested the changes with PyTorch 1.0.0 and Python 3.7 on MNIST and Freyfaces, giving me similar results for the baseline VAE without any flows.

I am not sure if more rigorous test should be done and if you want to merge this into master or keep a separate branch.

Best, Martin

Hi Rianne,

Thank you for this really nice code release :)

I cloned the repo and made some changes so that it runs with PyTorch 1.+ and Python 3. Also solved the issue mentioned in #1 . I tested the changes on MNIST (binary input) and Freyfaces (multinomial input), giving similar results to the original code.

If you are interested in reviewing and potentially adding this to the repo, I would be happy to clean things up and make a PR.

Best, Martin

Hi Rianne,

I'm trying to run the default experiment on cpu with a small latent space dimension (z=5):

python main_experiment.py -d mnist --flow no_flow -nc --z_size 5

Which unfortunately gives the following error:

Traceback (most recent call last):
  File "main_experiment.py", line 278, in <module>
    run(args, kwargs)
  File "main_experiment.py", line 189, in run
    tr_loss = train(epoch, train_loader, model, optimizer, args)
  File ".../sylvester-flows/optimization/training.py", line 39, in train
    loss.backward()
  File "//anaconda/envs/dl/lib/python3.6/site-packages/torch/tensor.py", line 102, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "//anaconda/envs/dl/lib/python3.6/site-packages/torch/autograd/__init__.py", line 90, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation

I am using PyTorch version 1.0.0 and did not modify the code.

Hello,

I was wondering how I can generate samples using the decoder network after training. In a VAE, I would just sample from the prior distribution z~N(0,1) and generate a data point using the decoder. In TriangularSylvesterVAE, however, I also have to provide hyperparameters lambda(x) that depend on the input. How can I sample from my latent distribution and generate samples from it?

I am new to normalizing flows in general and would appreciate any help.