Latent Space Symmetry Discovery

Equivariant neural networks require explicit knowledge of the symmetry group. Automatic symmetry discovery methods aim to relax this constraint and learn invariance and equivariance from data. However, existing symmetry discovery methods are limited to linear symmetries in their search space and cannot handle the complexity of symmetries in real-world, often high-dimensional data. We propose a novel generative model, Latent LieGAN (LaLiGAN), which can discover nonlinear symmetries from data. It learns a mapping from data to a latent space where the symmetries become linear and simultaneously discovers symmetries in the latent space. Theoretically, we show that our method can express any nonlinear symmetry under certain conditions. Experimentally, our method can capture the intrinsic symmetry in high-dimensional observations, which results in a well-structured latent space that is useful for other downstream tasks. We demonstrate the use cases for LaLiGAN in improving equation discovery and long-term forecasting for various dynamical systems. However, for complex real-world data, the underlying symmetries may be unknown or challenging to articulate through programming. Each et al., 2023) can discover various types of symmetries, trajectory is a group action orbit containing but its search space is still constrained to general linear a random v V. groups. Successful discovery can only be achieved when observations are measured in an ideal coordinate system where linear symmetry is present. Unfortunately, real-world data often contain nonlinear symmetries, such as high-dimensional dynamics that evolve on a low-dimensional manifold (Champion et al., 2019), or 2D images of 3D objects (Garrido et al., 2023). Another line of study focuses on learning equivariant representations (Park et al., 2022; Yu et al., 2022; Dangovski et al., 2021; Quessard et al., 2020). These approaches learn a latent embedding space with particular symmetries.