Sobolev Norm Learning Rates for Conditional Mean Embeddings

In the past decade, several studies have explored a new framework for embedding conditional distributions in reproducing kernel Hilbert spaces (RKHS). This approach seeks to represent a conditional distribution as an RKHS element, and thereby reduce the computation of conditional expectations to the evaluation of kernel inner products. Unlike other distribution learning approaches, which often involve density estimation and expensive numerical analysis, the conditional mean embedding (CME) framework exploits the popular kernel trick to allow distributions to be learned directly and efficiently from sample information, and do not require the target distribution to possess a density function. The broad generalizability and computational levity of conditional embeddings have led them to find many applications in reinforcement learning, hypothesis testing, and nonparametric inference [3-5, 17], where conditional relationships are often of pertinent interest. A central issue involved in the conditional embedding framework is the performance of the sample estimator. Despite their successful application, there has been a limited study of optimal learning rates for conditional mean embeddings. Several foundational works [17, 18] established the consistency of the sample embedding estimator, exploring its convergence rate to a "true" embedding in the RKHS norm. These works framed the act of conditioning as a linear operator between two Hilbert spaces, which mapped features of the independent variable in the input space to the mean embeddings of their respective conditional distributions in the output feature space.