Learning In-between Imagery Dynamics via Physical Latent Spaces

Understanding image dynamics from a set of complex measurement data is important in many applications, from the diagnosis or monitoring of a disease done by analyzing a series of medical (e.g. MRI or ultrasound) images, [28], to the interpretation of a sequence of satellite images used to study climate changes, natural disaster, or environmental conditions [2]. Here an "image" refers to a high-dimensional data frame that contains complex and condensed information within each pixel where these pixels are also spatially correlated. To understand the underlying dynamics between sequential images, therefore, it is essential to simultaneously decipher the intertwined relationship among their spatial and temporal features. A common approach for understanding such spatio-temporal dynamics involves the employment of physical models such as differential equations (DEs). By using the observed data to estimate the parameters in these corresponding DEs, it is possible to gain physical insights regarding their evolution [12, 20]. However, directly applying such techniques to image dynamics is of limited use due to the intricate description that would be required by a suitable prior model, the highly nonlinear relationship among pixels, and the computational complexities arising from the high dimensionality of the images.