Among them Nesterov's accelerated gradient algorithm [Nesterov, 1983] is proven to be optimal on the

A.2 Proof of Theorem 1 We restate the theorem for completeness: Theorem 1. Assume Any ODE's solution, if it exists and converges, converges to an's estimate of the conditional effect is We now bound the remaining term. 's computation of the surrogate intervention involved Thus, such error does not accumulate even with large step sizes. Theorem 4. Effect Connectivity is necessary for nonparametric effect estimation in Let Effect Connectivity be violated, i.e. there exists a Thus, nonparametric effect estimation is impossible. The effect threshold here is 0.1.Figure 7: True positive vs. False negative rate as we vary the threshold on average

This paper presents a new gradient flow dissipation geometry over non-negative and probability measures.

Otto's (2001) Wasserstein gradient flow of the exclusive KL divergence functional provides a powerful and mathematically principled perspective for analyzing learning and inference algorithms. In contrast, algorithms for the inclusive KL inference, i.e., minimizing $ \mathrm{KL}(\pi \| \mu) $ with respect to $ \mu $ for some target $ \pi $, are rarely analyzed using tools from mathematical analysis. This paper shows that a general-purpose approximate inclusive KL inference paradigm can be constructed using the theory of gradient flows derived from PDE analysis. We uncover that several existing learning algorithms can be viewed as particular realizations of the inclusive KL inference paradigm. For example, existing sampling algorithms such as Arbel et al. (2019) and Korba et al. (2021) can be viewed in a unified manner as inclusive-KL inference with approximate gradient estimators. Finally, we provide the theoretical foundation for the Wasserstein-Fisher-Rao gradient flows for minimizing the inclusive KL divergence.

We show that accelerated optimization methods can be seen as particular instances of multi-step integration schemes from numerical analysis, applied to the gradient flow equation. Compared with recent advances in this vein, the differential equation considered here is the basic gradient flow, and we derive a class of multi-step schemes which includes accelerated algorithms, using classical conditions from numerical analysis. Multi-step schemes integrate the differential equation using larger step sizes, which intuitively explains the acceleration phenomenon.

We consider the Wasserstein metric on the Gaussian mixture models (GMMs), which is defined as the pullback of the full Wasserstein metric on the space of smooth probability distributions with finite second moment. It derives a class of Wasserstein metrics on probability simplices over one-dimensional bounded homogeneous lattices via a scaling limit of the Wasserstein metric on GMMs. Specifically, for a sequence of GMMs whose variances tend to zero, we prove that the limit of the Wasserstein metric exists after certain renormalization. Generalizations of this metric in general GMMs are established, including inhomogeneous lattice models whose lattice gaps are not the same, extended GMMs whose mean parameters of Gaussian components can also change, and the second-order metric containing high-order information of the scaling limit. We further study the Wasserstein gradient flows on GMMs for three typical functionals: potential, internal, and interaction energies. Numerical examples demonstrate the effectiveness of the proposed GMM models for approximating Wasserstein gradient flows.

Partial differential equations are important tools in solving a wide range of problems in science and engineering fields. Over the past twenty years, deep neural networks (DNNs) [12, 19] have demonstrated their power in science and engineering applications, and efforts have been made to employ DNNs to solve complex partial differential equations as an alternative to the traditional numerical schemes, especially for problems in high dimensions. Early works [5, 17] use feedforward neural network to learn the initial/boundary value problem by constraining neural networks using differential equation. Methods using continuous dynamical systems to model high-dimensional nonlinear functions used in machine learning were proposed in [6]. A deep learning-based approach to solve high dimensional parabolic partial differential equations (PDEs) based on the formulation of stochastic differential equations was developed in [14].

Causal inference relies on two fundamental assumptions: ignorability and positivity. We study causal inference when the true confounder value can be expressed as a function of the observed data; we call this setting estimation with functional confounders (EFC). In this setting, ignorability is satisfied, however positivity is violated, and causal inference is impossible in general. We consider two scenarios where causal effects are estimable. First, we discuss interventions on a part of the treatment called functional interventions and a sufficient condition for effect estimation of these interventions called functional positivity. Second, we develop conditions for nonparametric effect estimation based on the gradient fields of the functional confounder and the true outcome function. To estimate effects under these conditions, we develop Level-set Orthogonal Descent Estimation (LODE). Further, we prove error bounds on LODE's effect estimates, evaluate our methods on simulated and real data, and empirically demonstrate the value of EFC.