Spiking neural networks have attracted increasing attention in recent years due to their potential of handling time-dependent data.

Spiking neural networks have attracted increasing attention in recent years due to their potential of handling time-dependent data. Many algorithms and techniques have been developed; however, theoretical understandings of many aspects of spiking neural networks are far from clear. A recent work [ 44 ] disclosed that typical spiking neural networks could hardly work on spatio-temporal data due to their bifurcation dynamics and suggested that the self-connection structure has to be added. In this paper, we theoretically investigate the approximation ability and computational efficiency of spiking neural networks with self connections, and show that the self-connection structure enables spiking neural networks to approximate discrete dynamical systems using a polynomial number of parameters within polynomial time complexities. Our theoretical results may shed some insight for the future studies of spiking neural networks.

The rise of deep learning challenges the longstanding scientific ideal of insight - the human capacity to understand phenomena by uncovering underlying mechanisms. In many modern applications, accurate predictions no longer require interpretable models, prompting debate about whether explainability is a realistic or even meaningful goal. From our perspective in physics, we examine this tension through a concrete case study: a physics-informed neural network (PINN) trained on a rarefied gas dynamics problem governed by the Boltzmann equation. Despite the system's clear structure and well-understood governing laws, the trained network's weights resemble Gaussian-distributed random matrices, with no evident trace of the physical principles involved. This suggests that deep learning and traditional simulation may follow distinct cognitive paths to the same outcome - one grounded in mechanistic insight, the other in statistical interpolation. Our findings raise critical questions about the limits of explainable AI and whether interpretability can - or should-remain a universal standard in artificial reasoning.

Machine Learning (ML) has taken science (and society) by storm in the last decade, with numerous applications which seem to defy our best theoretical and modeling tools [11]. Leaving aside a significant amount of hype, ML raises a number of genuine hopes to counter some the most vexing challenges for the scientific method, particularly the curse of dimensionality [2]. This however does not come for free; in particular the current trends towards the use of an astronomical number of parameters (trillions in the case of recent chatbots), none of which lends itself to a direct physical interpretation, jointly with an unsustainable power demand, beg for a change of strategy, namely less weights and more insight [17]. In this paper, we present an attempt along this line. In particular, by highlighting the one-to-one mapping between ML procedures and discrete dynamical systems, we suggest that ML could possibly be conducted by means of a restricted and more economical class of weight matrices, each of which can be interpreted as a specific information-propagation process.

We consider arbitrary bounded discrete time series originating from dynamical system with recursivity. More precisely, we provide an explicit construction of recurrent neural networks which effectively approximate the corresponding discrete dynamical systems.

All the solutions were probability distributions, and in this article we introduce an even larger, generic class of problems (chaotic discrete dynamical systems) with known solution. Each dynamical system discussed here (or in my previous article) comes with two distributions: The name Hurwitz and Riemann-Zeta is just a reminder of their strong connection to number theory problems such as continued fractions, approximation of irrational numbers by rational ones, the construction and distribution of the digits of random numbers in various numeration systems, and the famous Riemann Hypothesis that has a one million dollar prize attached to it. The most well known probability distribution related to these functions is the discrete Zipf distribution. The author defines a family of distribution that generalizes the exponential power, normal, gamma, Weibull, Rayleigh, Maxwell-Boltzmann and chi-squared distributions, with applications in actuarial sciences. Our Hurwitz-Riemann Zeta distribution is yet another example arising this time from discrete dynamical systems, continuous on [0, 1].

Each training step for a variational autoencoder (VAE) requires us to sample from the approximate posterior, so we usually choose simple (e.g. factorised) approximate posteriors in which sampling is an efficient computation that fully exploits GPU parallelism. However, such simple approximate posteriors are often insufficient, as they eliminate statistical dependencies in the posterior. While it is possible to use normalizing flow approximate posteriors for continuous latents, some problems have discrete latents and strong statistical dependencies. The most natural approach to model these dependencies is an autoregressive distribution, but sampling from such distributions is inherently sequential and thus slow. We develop a fast, parallel sampling procedure for autoregressive distributions based on fixed-point iterations which enables efficient and accurate variational inference in discrete state-space latent variable dynamical systems. To optimize the variational bound, we considered two ways to evaluate probabilities: inserting the relaxed samples directly into the pmf for the discrete distribution, or converting to continuous logistic latent variables and interpreting the K-step fixed-point iterations as a normalizing flow. We found that converting to continuous latent variables gave considerable additional scope for mismatch between the true and approximate posteriors, which resulted in biased inferences, we thus used the former approach. Using our fast sampling procedure, we were able to realize the benefits of correlated posteriors, including accurate uncertainty estimates for one cell, and accurate connectivity estimates for multiple cells, in an order of magnitude less time.