determinant
- Europe > United Kingdom > England > Cambridgeshire > Cambridge (0.14)
- North America > United States > Maryland > Baltimore (0.04)
- North America > United States > Hawaii > Honolulu County > Honolulu (0.04)
- (5 more...)
- North America > United States > California > Orange County > Irvine (0.04)
- Europe > Spain > Catalonia > Barcelona Province > Barcelona (0.04)
- Europe > Netherlands > North Holland > Amsterdam (0.04)
- Asia > Middle East > Jordan (0.04)
- North America > United States > New York > Tompkins County > Ithaca (0.04)
- North America > Canada > Quebec > Montreal (0.04)
- Europe > Germany > Hesse > Darmstadt Region > Wiesbaden (0.04)
Appendix for: Invertible Gaussian Reparameterization 1 Computing the determinant of the Jacobian of the softmax + +
As mentioned in section 3.1, we can use the matrix determinant lemma to efficiently compute the Proof: For k = 1,...,K 1, we have: P(H = k) = null As mentioned in the main manuscript, our V AE experiments closely follow Maddison et al. RELAX builds upon equation 16 to develop an estimator with reduced variance. Third, it should also be noted that the bias and variance of the gradient estimator of RELAX are central points of discussion by Grathwohl et al. We show results of running IGR and GS with and without RELAX in Table 2. Discrete Models MNIST IGR-I -94.18 GS -103.80 IGR-I + RELAX -81.95 GS + RELAX -83.41 Table 2: Test log-likelihood on MNIST for nonlinear architecture. Results are in Table 3 and we can see that again, IGR performs best.
- North America > United States > Virginia (0.04)
- Europe > United Kingdom (0.04)
- Europe > Hungary > Budapest > Budapest (0.04)
- Europe > Belgium > Flanders (0.04)
- North America > United States > Colorado > Boulder County > Boulder (0.04)
- Europe > Netherlands > North Holland > Amsterdam (0.04)
- North America > United States > New York > Tompkins County > Ithaca (0.04)
- North America > Canada > Quebec > Montreal (0.04)
- Europe > Germany > Hesse > Darmstadt Region > Wiesbaden (0.04)
CDFlow: Building Invertible Layers with Circulant and Diagonal Matrices
Normalizing flows are deep generative models that enable efficient likelihood estimation and sampling through invertible transformations. A key challenge is to design linear layers that enhance expressiveness while maintaining efficient computation of the Jacobian determinant and inverse. We introduce a novel invertible linear layer based on the product of circulant and diagonal matrices. This decomposition reduces parameter complexity from $\mathcal{O}(n^2)$ to $\mathcal{O}(mn)$ using $m$ diagonal matrices and $m-1$ circulant matrices while still approximating general linear transformations. By leveraging the Fast Fourier Transform, our approach reduces the time complexity of matrix inversion from $\mathcal{O}(n^3)$ to $\mathcal{O}(mn\log n)$ and that of computing the log-determinant from $\mathcal{O}(n^3)$ to $\mathcal{O}(mn)$, where $n$ is the input dimension. We build upon this layer to develop Circulant-Diagonal Flow (CDFlow), which achieves strong density estimation on natural image datasets and effectively models data with inherent periodic structure. Furthermore, CDFlow significantly accelerates key operations in normalizing flows, providing practical benefits for scalable generative modeling.
- North America > United States (0.14)
- Asia > China (0.04)