A V Experiments

The hyperparameters for these experiments are discussed in C. In all experiments, we use the same V AE-architecture as [58], and reshape the images to be 32 32 . The size of the latent dimension is set to be 50 therefore the maximum # of Active Units is 50 . The metrics considered to measure the performance of V AEs include Negative Log-Likelihood, # of Active Units in the latent space and the Mutual Information between the input x and the latent space z, I ( z; x). We use the same formulation as [7] to compute NLL and # of Active Units, and use the same formulation as [9, 21] to approximate the Mutual Information. Bolded value corresponds to the proposed model which uses the kernel layer after each convolutional layer.