Variational autoencoders (VAEs) are one of the powerful likelihood-based generative models with applications in many domains.

Infact, bysubstitutingp =1n into Equation (8), weobtain: Proposition 5.For q = ( c,..., c, 0) Rn withc = 2 Δ logn, theexpectederrorE [ E ( MPF, q)] ofpermute-and-flipisatleastΔ2 log ( n).

In other cases, the motivation may simply be to train more reliable classifiers.

Geometric embeddings have recently received attention for their natural ability to represent transitive asymmetric relations via containment. Box embeddings, whereobjectsarerepresentedby n-dimensionalhyperrectangles,areaparticularly promising example of such an embedding as they are closed under intersection and their volume can be calculated easily, allowing them to naturally represent calibrated probability distributions.

It can be inferred from Definition 1 that all elements in the definition of a ReLU network training problem are piecewise smooth, where each piece is an elementary log exp function. We refer the reader to [30] for an introduction to piecewise smoothness and recent use of such notions in the context of algorithmic differentiation in [8]. Let us first argue that the results of [8] apply to Definition1. This is Theorem 2 for s [0,T], note that a similar probabilistic argument was developped in [6]. Consider any fully connected ReLU network architecture of depth H, with the softmax function appliedonthelastlayer.