Goto

Collaborating Authors

 discretization




A Constrained sampling via post-processed denoiser In this section, we provide more details on the apparatus necessary to perform a posteriori conditional

Neural Information Processing Systems

Eq. (6) suggests that the SDE drift corresponding to the score may be broken down into 3 steps: 1. However, in practice this modification creates a "discontinuity" between the constrained and unconstrained components, leading to erroneous correlations between them in the generated samples. "learning rate" that is determined empirically such that the loss value reduces adequately close to zero Thus it needs to be tuned empirically. The correction in Eq. (16) is equivalent to imposing a Gaussian likelihood on Remark 2. The post-processing presented in this section is similar to [ In this section, we present the most relevant components for completeness and better reproducibility. B.2 Sampling The reverse SDE in Eq. (5) used for sampling may be rewritten in terms of denoiser D As stated in 4.1 of the main text, for this The energy-based metrics are already defined in Eq. (12) and Eq.







A Theory of Diversity for Random Matrices with Applications to In-Context Learning of Schrödinger Equations

Cole, Frank, Lu, Yulong, Sehgal, Shaurya

arXiv.org Machine Learning

We address the following question: given a collection $\{\mathbf{A}^{(1)}, \dots, \mathbf{A}^{(N)}\}$ of independent $d \times d$ random matrices drawn from a common distribution $\mathbb{P}$, what is the probability that the centralizer of $\{\mathbf{A}^{(1)}, \dots, \mathbf{A}^{(N)}\}$ is trivial? We provide lower bounds on this probability in terms of the sample size $N$ and the dimension $d$ for several families of random matrices which arise from the discretization of linear Schrödinger operators with random potentials. When combined with recent work on machine learning theory, our results provide guarantees on the generalization ability of transformer-based neural networks for in-context learning of Schrödinger equations.


Manifold limit for the training of shallow graph convolutional neural networks

Tengler, Johanna, Brune, Christoph, Iglesias, José A.

arXiv.org Machine Learning

We study the discrete-to-continuum consistency of the training of shallow graph convolutional neural networks (GCNNs) on proximity graphs of sampled point clouds under a manifold assumption. Graph convolution is defined spectrally via the graph Laplacian, whose low-frequency spectrum approximates that of the Laplace-Beltrami operator of the underlying smooth manifold, and shallow GCNNs of possibly infinite width are linear functionals on the space of measures on the parameter space. From this functional-analytic perspective, graph signals are seen as spatial discretizations of functions on the manifold, which leads to a natural notion of training data consistent across graph resolutions. To enable convergence results, the continuum parameter space is chosen as a weakly compact product of unit balls, with Sobolev regularity imposed on the output weight and bias, but not on the convolutional parameter. The corresponding discrete parameter spaces inherit the corresponding spectral decay, and are additionally restricted by a frequency cutoff adapted to the informative spectral window of the graph Laplacians. Under these assumptions, we prove $Γ$-convergence of regularized empirical risk minimization functionals and corresponding convergence of their global minimizers, in the sense of weak convergence of the parameter measures and uniform convergence of the functions over compact sets. This provides a formalization of mesh and sample independence for the training of such networks.