Nonparametric models have extensive applications across diverse fields, including biology (Mac-Farland et al. (2016)), economics (Ullah and Pagan (1999); Li and Racine (2023)), engineering (Lanzante (1996)), and machine learning (Hofmann et al. (2008); Schmidt-hieber (2020)). The most representative nonparametric approaches are kernel methods, known for their numerical robustness and statistical stability in lower-dimensional settings. However, kernel methods often suffer from the curse of dimensionality in higher-dimensional spaces. Recently, a number of significant studies have tackled various modern challenges in nonparametric models. For example, Ravikumar et al. (2009), Raskutti et al. (2012), and Yuan and Zhou (2016) have studied additive models for high-dimensional nonparametric regression; Zhang et al. (2015) and Yang et al. (2017) analyzed randomized algorithms for kernel regression estimation; and Liu et al. (2007) explored nonparametric density estimation in higher dimensions. Despite these contributions, the curse of dimensionality in nonparametric problems, particularly in aspects of statistical accuracy and computational efficiency, remains an open area for further exploration. In this paper, we aim to develop a new framework specifically designed for nonparametric estimation problems. Within this framework, we conceptualize functions as matrices or tensors and explore new methods for handling the bias-variance trade-off, aiming to reduce the curse of dimensionality in higher dimensions. Matrix approximation algorithms, such as singular value decomposition and QR decomposition, play a crucial role in computational mathematics and statistics.

In this paper, we introduce and study matching methods based on distance profiles. For the matching of point clouds, the proposed method is easily implementable by solving a linear program, circumventing the computational obstacles of quadratic matching. Also, we propose and analyze a flexible way to execute location-to-location matching using distance profiles. Moreover, we provide a statistical estimation error analysis in the context of location-to-location matching using empirical process theory. Furthermore, we apply our method to a certain model and show its noise stability by characterizing conditions on the noise level for the matching to be successful. Lastly, we demonstrate the performance of the proposed method and compare it with some existing methods using synthetic and real data.

In this paper, we propose a general framework for solving high-dimensional partial differential equations with tensor networks. Our approach uses Monte-Carlo simulations to update the solution and re-estimates the new solution from samples as a tensor-network using a recently proposed tensor train sketching technique. We showcase the versatility and flexibility of our approach by applying it to two specific scenarios: simulating the Fokker-Planck equation through Langevin dynamics and quantum imaginary time evolution via auxiliary-field quantum Monte Carlo. We also provide convergence guarantees and numerical experiments to demonstrate the efficacy of the proposed method.

Fueled by the expressive power of deep neural networks, normalizing flows have achieved spectacular success in generative modeling, or learning to draw new samples from a distribution given a finite dataset of training samples. Normalizing flows have also been applied successfully to variational inference, wherein one attempts to learn a sampler based on an expression for the log-likelihood or energy function of the distribution, rather than on data. In variational inference, the unimodality of the reference Gaussian distribution used within the normalizing flow can cause difficulties in learning multimodal distributions. We introduce an extension of normalizing flows in which the Gaussian reference is replaced with a reference distribution that is constructed via a tensor network, specifically a matrix product state or tensor train. We show that by combining flows with tensor networks on difficult variational inference tasks, we can improve on the results obtained by using either tool without the other.

Orbit recovery problems are a class of problems that often arise in practice and in various forms. In these problems, we aim to estimate an unknown function after being distorted by a group action and observed via a known operator. Typically, the observations are contaminated with a non-trivial level of noise. Two particular orbit recovery problems of interest in this paper are multireference alignment and single-particle cryo-EM modeling. In order to suppress the noise, we suggest using the method of moments approach for both problems while introducing deep neural network priors. In particular, our neural networks should output the signals and the distribution of group elements, with moments being the input. In the multireference alignment case, we demonstrate the advantage of using the NN to accelerate the convergence for the reconstruction of signals from the moments. Finally, we use our method to reconstruct simulated and biological volumes in the cryo-EM setting.

We propose a hierarchical tensor-network approach for approximating high-dimensional probability density via empirical distribution. This leverages randomized singular value decomposition (SVD) techniques and involves solving linear equations for tensor cores in this tensor network. The complexity of the resulting algorithm scales linearly in the dimension of the high-dimensional density. An analysis of estimation error demonstrates the effectiveness of this method through several numerical experiments.

We propose the tensorizing flow method for estimating high-dimensional probability density functions from the observed data. The method is based on both tensor-train and flow-based generative modeling. Our method first efficiently constructs an approximate density in the tensor-train form via solving the tensor cores from a linear system based on the kernel density estimators of low-dimensional marginals. We then train a continuous-time flow model from this tensor-train density to the observed empirical distribution by performing a maximum likelihood estimation. The proposed method combines the optimization-less feature of the tensor-train with the flexibility of the flow-based generative models. Numerical results are included to demonstrate the performance of the proposed method.

Inverse wave scattering aims at determining the properties of an object using data on how the object scatters incoming waves. In order to collect information, sensors are put in different locations to send and receive waves from each other. The choice of sensor positions and incident wave frequencies determines the reconstruction quality of scatterer properties. This paper introduces reinforcement learning to develop precision imaging that decides sensor positions and wave frequencies adaptive to different scatterers in an intelligent way, thus obtaining a significant improvement in reconstruction quality with limited imaging resources. Extensive numerical results will be provided to demonstrate the superiority of the proposed method over existing methods.

Community detection and synchronization are both fundamental problems in signal processing, machine learning, and computer vision. Recently, there is an increasing interest in their joint problem [27, 8, 44]. That is, in the presence of heterogeneous data where data points associated with random group elements (e.g. the orthogonal group O(d) of dimension d) fall into multiple underlying clusters, the joint problem is to simultaneously recover the cluster structures as well as the group elements. A motivating example is the 2D class averaging process in cryo-electron microscopy single particle reconstruction [30, 58, 68], whose goal is to align (with SO(2) group synchronization) and average projection images of a single particle with similar viewing angles to improve their signal-to-noise ratio (SNR). Another application in computer vision is simultaneous permutation group synchronization and clustering on heterogeneous object collections consisting of 2D images or 3D shapes [8]. In this work, we study the joint problem based on the probabilistic model introduced in [27] which extends the celebrated stochastic block model (SBM) [19, 21, 22, 29, 38, 41, 49, 50, 51, 52] (see Figure 1) for community detection. In particular, we focus on the orthogonal group O(d) that covers a wide range of applications mentioned above.

In the presence of heterogeneous data, where randomly rotated objects fall into multiple underlying categories, it is challenging to simultaneously synchronize the objects and classify them into clusters. A motivating example is the 2D class averaging in cryo-electron microscopy single particle reconstruction [17, 41, 49]. It aims to cluster images taken from similar viewing directions, rotationally align and average those images in order to denoise the experimental data. Joint clustering and synchronization is an emerging area that connects community detection [1, 16, 32, 2, 19, 20] and synchronization [39, 7, 18]. Recently, several works discussed simultaneous classification and mapping (alignment) [4, 24] and proposed different models and algorithms. In [4], the authors addressed simultaneous permutation group synchronization and clustering via a spectral method with rounding and used the consistency of the mapping for clustering. In [24], the authors noticed that both rotational alignment and classification are problems over compact groups and proposed a harmonic analysis and semidefinite programming based approach for solving alignment and classification simultaneously. In this paper, we consider joint community detection and rotational synchronization under a specific probabilistic model, which extends the celebrated stochastic block model (SBM) [9, 10, 11, 15, 21, 22, 28, 29, 30, 31] to incorporate both the community structure and pairwise rotations. In particular, we inherit the G(n, p, q)-SBM setting [26, 19, 2, 20] for the graph connection as shown in Figure 1.