Factorization with Uncertainty and Missing Data: Exploiting Temporal Coherence

The problem of \Structure From Motion" is a central problem in vision: given the 2D locations of certain points we wish to recover the camera motion and the 3D coordinates of the points. Un- der simplifled camera models, the problem reduces to factorizing a measurement matrix into the product of two low rank matrices. Each element of the measurement matrix contains the position of a point in a particular image. When all elements are observed, the problem can be solved trivially using SVD, but in any realistic sit- uation many elements of the matrix are missing and the ones that are observed have a difierent directional uncertainty. Under these conditions, most existing factorization algorithms fail while human perception is relatively unchanged.