Reviews: Orthogonally Decoupled Variational Gaussian Processes

I agree the standard error can be misleading due to the correlations induced by the datasets while the standard error of the ranks is more informative. After the discussion with other reviewers, I think adding the motivation (or derivation) for Equation 9 will definitely make the paper stronger and I encourage the authors to do so in their final version. This paper proposes a novel RKHS parameterization of decoupled GPs that admits efficient natural gradient computation. Specifically, they decompose the mean parameterization into a part that shares the basis with the covariances, and an orthogonal part that models the residues that the standard decoupled GP (Chen and Boots, 2017) fails to capture. This construction allows for a straightforward natural gradient update rule.