Kronecker Sum Decompositions of Space-Time Data

Abstract--In this paper we consider the use of the space vs. time Kronecker product decomposition in the estimation of covariance matrices for spatiotemporal data. This decomposition imposes lower dimensional structure on the estimated covariance matrix, thus reducing the number of samples required for estimation. T o allow a smooth tradeoff between the reduction in the number of parameters (to reduce estimation variance) and the accuracy of the covariance approximation (affecting estimation bias), we introduce a diagonally loaded modification of the sum-of-kronecker products representation in [1]. We derive an asymptotic Cram er-Rao bound (CRB) on the minimum attainable mean squared predictor coefficient estimation error for unbiased estimators of Kronecker structured covariance matrices. We illustrate the accuracy of the diagonally loaded Kronecker sum decomposition by applying it to the prediction of human activity video. In this paper, we develop a method for estimation of spatiotemporal covariance and apply it to video modeling and prediction. The covariance for spatiotemporal processes manifests itself as multiframe covariance, i.e. the covariance not only between pixels or features in a single frame, but also between pixels or features in a set of nearby frames.