Sublinear Time Orthogonal Tensor Decomposition

Song, Zhao, Woodruff, David, Zhang, Huan

Neural Information Processing Systems 

Their algorithm is based on computing sketches of the input tensor, which requires reading the entire input. We show in a number of cases one can achieve the same theoretical guarantees in sublinear time, i.e., even without reading most of the input tensor. Instead of using sketches to estimate inner products in tensor decomposition algorithms, we use importance sampling. To achieve sublinear time, we need to know the norms of tensor slices, and we show how to do this in a number of important cases. For the important case of p 3 and small values of k, we can also estimate such norms.