The discovery of governing differential equations from data is an open frontier in machine learning. The sparse identification of nonlinear dynamics (SINDy) \citep{brunton_discovering_2016} framework enables data-driven discovery of interpretable models in the form of sparse, deterministic governing laws. Recent works have sought to adapt this approach to the stochastic setting, though these adaptations are severely hampered by the curse of dimensionality. On the other hand, Bayesian-inspired deep learning methods have achieved widespread success in high-dimensional probabilistic modeling via computationally efficient approximate inference techniques, suggesting the use of these techniques for efficient stochastic equation discovery. Here, we introduce HyperSINDy, a framework for modeling stochastic dynamics via a deep generative model of sparse governing equations whose parametric form is discovered from data. HyperSINDy employs a variational encoder to approximate the distribution of observed states and derivatives. A hypernetwork \citep{ha_hypernetworks_2016} transforms samples from this distribution into the coefficients of a differential equation whose sparse form is learned simultaneously using a trainable binary mask \citep{louizos_learning_2018}. Once trained, HyperSINDy generates stochastic dynamics via a differential equation whose coefficients are driven by a Gaussian white noise. In experiments, HyperSINDy accurately recovers ground truth stochastic governing equations, with learned stochasticity scaling to match that of the data. Finally, HyperSINDy provides uncertainty quantification that scales to high-dimensional systems. Taken together, HyperSINDy offers a promising framework for model discovery and uncertainty quantification in real-world systems, integrating sparse equation discovery methods with advances in statistical machine learning and deep generative modeling.

The main goal of the DGM to estimate the likelihood of each observation and to create new samples from the underlying distribution, in other words take as input training samples from some distribution and learn a model that represents the distribution.It is achieved by density estimation and sample generation. We will be having some data which has a probability distribution and when we give new data this generates samples with the reference of the input PDF. Variables that are not directly observed but rather inferred from other variables that are observed(true exploratory variables).In latent variable model, there are different types of models auto encoders, Variational Auto encoders(VAEs) and Generative adversarial networks(GANs) Foundational generative model which tries to build this latent Variable Representation by self encoding the inputs.Encoder,Learns mapping from the data X to low dimensional latent space Z. Decode - mapping back from latent space Z to reconstruct observation as X' Autoencoders focus on Bottleneck layer which helps to learn a compressed latent representation and Reconstruction loss which helps the latent representation to capture(or encode) as much "information" about the data as possible. In this loss function does not use any labels, more likely output minus input is the loss obtained L(x) x-x' ² For VAEs impose a stochastic or variational twist on this architecture helps to generate smoother representation of data.In this instead of learning the latent variable(z) directly for each variable the VAEs learns mean and variance associated with that latent variable.this With the help of regularization term we can avoid overfitting.Futher Unlike in Autoencoders, In VAEs we cannot directly back-propagate gradients through sampling layers.To achieve back-propagation we need to parameterise the sampling layers such that, it could be trained end to end.

Deep generative models (DGM) are neural networks with many hidden layers trained to approximate complicated, high-dimensional probability distributions using a large number of samples. When trained successfully, we can use the DGMs to estimate the likelihood of each observation and to create new samples from the underlying distribution. Developing DGMs has become one of the most hotly researched fields in artificial intelligence in recent years. The literature on DGMs has become vast and is growing rapidly. Some advances have even reached the public sphere, for example, the recent successes in generating realistic-looking images, voices, or movies; so-called deep fakes.