Simulation-based techniques such as variants of stochastic Runge-Kutta are thede facto approach for inference with stochastic differential equations (SDEs) in machine learning. These methods are general-purpose and used with parametric and non-parametric models, and neural SDEs. Stochastic Runge-Kutta relies on the use of sampling schemes that can be inefficient in high dimensions. We addressthis issue by revisiting the classical SDE literature and derive direct approximations to the (typically intractable) Fokker-Planck-Kolmogorov equation by matchingmoments. We show how this workflow is fast, scales to high-dimensional latent spaces, and is applicable to scarce-data applications, where a non-parametric SDE with a driving Gaussian process velocity field specifies the model.

The introduction of convolution and attention to the space of rays in 3D required additional geometric representations for which there was no space in the main paper to elaborate. We will introduce here all the necessary notations and definitions. We have accompanied this presentation with examples of specific groups to elucidate the abstract concepts needed in the definitions. Figure 10: The visualization of Plücker coordinates: A ray xcan be denoted as (d,m)where x is any point on the ray x, and dis the direction of the ray x. mis defined as x d. Given the action of the group G on a homogeneous space X, and given x0 as the origin of X, the stabilizer group H of x0 in G is the group that leaves x0 intact, i.e., H = {h G|hx0 = x0}. The group, G, can be partitioned into the quotient space (the set of left cosets) G/H and X is isomorphic to G/H since all group elements in the same coset transform x0 to the same element in X, that is, for any element g gH we have g x0 = gx0. Example 1. SE(3) acting on the ray space R: Take SE(3) as the acting group and the ray space R as its homogeneous space. We use Plücker coordinates to parameterize the ray space R: any x R can be denoted as (d,m), where d S2 is the direction of the ray, and m = x d where x is any point on the ray, as shown in figure 10. R is the quotient space SE(3)/(SO(2) R)up to isomorphism. Example 2. SE(3) acting on the 3DEuclidean space R3: R3 is isomorphic to SE(3)/SO(3). Consider another case when SE(3) acts on the homogeneous space R3; for any g = (R,t) SE(3) and x R3, gx = Rx+t. If the fixed origin is [0,0,0]T, the stabilizer subgroup is H = SO(3) since any rotation g = (R,0)leaves [0,0,0]T unchanged. The last example is SO(3) acting on the homogeneous space sphere S2. Given the fixed origin point as [0,0,1]T, the stabilizer group is SO(2).

A.1 Proof of Theorem 2 Consider an instance of semi-supervised active regression with the labeled dataset Xlab being empty, and with the unlabeled dataset composed of k copies of the standard basis vector ei for each i = 1,,d.

Labeled data often comes at a high cost as it may require recruiting human labelers or running costly' experiments. At the same time, in many practical scenarios, one already has access to a partially labeled, potentially biased dataset that can help with the learning task at hand. Motivated by such settings, we formally initiate a study of semi-supervised active learning through the frame of linear regression.