On the Value of Infinite Gradients in Variational Autoencoder Models

But it remains an open question: What might the unintended consequences of such a restriction be? To address this issue, we examine how unbounded gradients relate to the regularization of a broad class of autoencoder-based architectures, including V AE models, as applied to data lying on or near a low-dimensional manifold (e.g., natural images).