We consider classical probabilistic principal component analysis ( PPCA) and show that its local latent variables can suffer from posterior collapse at maximum likelihood parameter values (i.e. 's are the latent variables of interest and others's are not (fully) identifiable in this However, it is nearly non-identifiable. While the two data generating clusters are different, they are very similar to each other because they overlap. We first define the most general form of LIDV AE . The key difference is in Eq. 19, where the classical VA E uses an arbitrary function General LIDV AE emulate many existing VA E . This general LIDV AE also subsumes the Bernoulli mixture model, which is a common variant of LIDGMV AE for the MNIST data. Moreover, for any data distribution generated by the classical VA E ( Eqs. 17 to 19), there exists an LIDV AE that can generate the same distribution.

While these approaches learn high-quality representations for downstream tasks such as action recognition, their training objectives disregard spatial cues naturally occurring in audio and visual signals.

The NFM framework utilizes the classical idea of multiplicative frailty in survival analysis as a principled way of extending the proportional hazard assumption, at the same time being able to leverage the strong approximation power of neural architectures for handling nonlinear covariate dependence.