stochastic resonance
Self-induced stochastic resonance: A physics-informed machine learning approach
Savaliya, Divyesh, Yamakou, Marius E.
Self-induced stochastic resonance (SISR) is the emergence of coherent oscillations in slow-fast excitable systems driven solely by noise, without external periodic forcing or proximity to a bifurcation. This work presents a physics-informed machine learning framework for modeling and predicting SISR in the stochastic FitzHugh-Nagumo neuron. We embed the governing stochastic differential equations and SISR-asymptotic timescale-matching constraints directly into a Physics-Informed Neural Network (PINN) based on a Noise-Augmented State Predictor architecture. The composite loss integrates data fidelity, dynamical residuals, and barrier-based physical constraints derived from Kramers' escape theory. The trained PINN accurately predicts the dependence of spike-train coherence on noise intensity, excitability, and timescale separation, matching results from direct stochastic simulations with substantial improvements in accuracy and generalization compared with purely data-driven methods, while requiring significantly less computation. The framework provides a data-efficient and interpretable surrogate model for simulating and analyzing noise-induced coherence in multiscale stochastic systems.
A Multiscale Approach for Enhancing Weak Signal Detection
Vimalajeewa, Dixon, Muller, Ursula U., Vidakovic, Brani
Stochastic resonance (SR), a phenomenon originally introduced in climate modeling, enhances signal detection by leveraging optimal noise levels within non-linear systems. Traditional SR techniques, mainly based on single-threshold detectors, are limited to signals whose behavior does not depend on time. Often large amounts of noise are needed to detect weak signals, which can distort complex signal characteristics. To address these limitations, this study explores multi-threshold systems and the application of SR in multiscale applications using wavelet transforms. In the multiscale domain signals can be analyzed at different levels of resolution to better understand the underlying dynamics. We propose a double-threshold detection system that integrates two single-threshold detectors to enhance weak signal detection. We evaluate it both in the original data domain and in the multiscale domain using simulated and real-world signals and compare its performance with existing methods. Experimental results demonstrate that, in the original data domain, the proposed double-threshold detector significantly improves weak signal detection compared to conventional single-threshold approaches. Its performance is further improved in the frequency domain, requiring lower noise levels while outperforming existing detection systems. This study advances SR-based detection methodologies by introducing a robust approach to weak signal identification, with potential applications in various disciplines.
Test-Time Defense Against Adversarial Attacks via Stochastic Resonance of Latent Ensembles
Lao, Dong, Zhang, Yuxiang, Oskouie, Haniyeh Ehsani, Wu, Yangchao, Wong, Alex, Soatto, Stefano
We propose a test-time defense mechanism against adversarial attacks: imperceptible image perturbations that significantly alter the predictions of a model. Unlike existing methods that rely on feature filtering or smoothing, which can lead to information loss, we propose to "combat noise with noise" by leveraging stochastic resonance to enhance robustness while minimizing information loss. Our approach introduces small translational perturbations to the input image, aligns the transformed feature embeddings, and aggregates them before mapping back to the original reference image. This can be expressed in a closed-form formula, which can be deployed on diverse existing network architectures without introducing additional network modules or fine-tuning for specific attack types. The resulting method is entirely training-free, architecture-agnostic, and attack-agnostic. Empirical results show state-of-the-art robustness on image classification and, for the first time, establish a generic test-time defense for dense prediction tasks, including stereo matching and optical flow, highlighting the method's versatility and practicality. Specifically, relative to clean (unperturbed) performance, our method recovers up to 68.1% of the accuracy loss on image classification, 71.9% on stereo matching, and 29.2% on optical flow under various types of adversarial attacks. Most deep neural networks in use today are deterministic maps from a fixed-size input to a fixed-size feature vector. In either case, the output vector is often highly sensitive to perturbations of the input, and one can intentionally choose these imperceptible perturbations adversarially so as to maximize the change in the output Goodfellow et al. (2014). In some cases, the same perturbation can even be disruptive for a large number of possible inputs Moosavi-Dezfooli et al. (2017), exploiting the convoluted geometry of the decision boundary imposed by such trained models Tram ` er et al. (2017). This spurious sensitivity could be exploited adversarially to disrupt the operation of a model.
Stochastic Resonance Improves the Detection of Low Contrast Images in Deep Learning Models
Stochastic resonance describes the utility of noise in improving the detectability of weak signals in certain types of systems. It has been observed widely in natural and engineered settings, but its utility in image classification with rate-based neural networks has not been studied extensively. In this analysis a simple LSTM recurrent neural network is trained for digit recognition and classification. During the test phase, image contrast is reduced to a point where the model fails to recognize the presence of a stimulus. Controlled noise is added to partially recover classification performance. The results indicate the presence of stochastic resonance in rate-based recurrent neural networks.
Harnessing the Power of Noise: A Survey of Techniques and Applications
Abdolazimi, Reyhaneh, Jin, Shengmin, Varshney, Pramod K., Zafarani, Reza
In Computer science and across various engineering fields, noise is often considered a nuisance and annoyance. It distorts details and makes data less accurate. In the past, the goal has often been to eliminate noise with the goal to make systems more reliable and accurate. But views on noise are changing. New findings suggest that noise can actually enhance and advance technologies in many areas, making us see it not just as a disruption but as a way to improve system performance. Thus, once unwanted and hard to control, noise now appears to be a key player in improving the performance of complex information processing systems [22]. This phenomena is often known as Stochastic Resonance, which helps clear up signals, improve image quality, and strengthen models in machine learning [7, 22, 101]. This duality of noise -- both a problem and a benefit -- highlights the tricky role of noise while optimizing advanced neural networks and machine learning models.
Emergence of a stochastic resonance in machine learning
Zhai, Zheng-Meng, Kong, Ling-Wei, Lai, Ying-Cheng
Department of Physics, Arizona State University, Tempe, Arizona 85287, USA (Dated: November 21, 2022) Can noise be beneficial to machine-learning prediction of chaotic systems? Utilizing reservoir computers as a paradigm, we find that injecting noise to the training data can induce a stochastic resonance with significant benefits to both short-term prediction of the state variables and longterm prediction of the attractor of the system. A key to inducing the stochastic resonance is to include the amplitude of the noise in the set of hyperparameters for optimization. By so doing, the prediction accuracy, stability and horizon can be dramatically improved. The stochastic resonance phenomenon is demonstrated using two prototypical high-dimensional chaotic systems. The interplay between noise and nonlinear dynamics have revealed that, if the hyperparameters are often leads to surprising phenomena with potentially significant not optimized, noise in the training data can improve applications and thus has always been an active to certain extent the prediction performance.
Noise Is a Drug and New York Is Full of Addicts - Issue 46: Balance
As soon as the door slams, I slide to the floor in a cross-legged position and hold my breath. The room in which I have just barricaded myself looks a bit like Matilda's chokey; a single light bulb casts a sickly yellow glow about the room, its walls lined with triangle-shaped chunks of fiberglass straining against wire mesh. In 15 minutes I will leave this room for the cacophonous world of Manhattan. I should, theoretically, be appreciating this small respite for what it is. Even so, with every second, I feel as if I'm going deeper underwater. I am sitting in an anechoic chamber, the only one in New York City. Nestled in the hip, angled building of The Cooper Union for the Advancement of Science and Art, the anechoic chamber is where acoustics students, headed by the aptly-named Melody Baglione, conduct research--it's the equivalent of a zero-gravity chamber, only in this case, the variable is sound. The room is designed to be as noise-free as possible; its chunky walls completely absorb reflections of sound waves, and insulate the space within from all exterior sources of noise.
Activity Driven Adaptive Stochastic Resonance
Wenning, Gregor, Obermayer, Klaus
Cortical neurons might be considered as threshold elements integrating in parallel many excitatory and inhibitory inputs. Due to the apparent variability of cortical spike trains this yields a strongly fluctuating membrane potential, such that threshold crossings are highly irregular. Here we study how a neuron could maximize its sensitivity w.r.t. a relatively small subset of excitatory input. Weak signals embedded in fluctuations is the natural realm of stochastic resonance. The neuron's response is described in a hazard-function approximation applied to an Ornstein-Uhlenbeck process.
Activity Driven Adaptive Stochastic Resonance
Wenning, Gregor, Obermayer, Klaus
Cortical neurons might be considered as threshold elements integrating in parallel many excitatory and inhibitory inputs. Due to the apparent variability of cortical spike trains this yields a strongly fluctuating membrane potential, such that threshold crossings are highly irregular. Here we study how a neuron could maximize its sensitivity w.r.t. a relatively small subset of excitatory input. Weak signals embedded in fluctuations is the natural realm of stochastic resonance. The neuron's response is described in a hazard-function approximation applied to an Ornstein-Uhlenbeck process.
Activity Driven Adaptive Stochastic Resonance
Wenning, Gregor, Obermayer, Klaus
Cortical neurons might be considered as threshold elements integrating inparallel many excitatory and inhibitory inputs. Due to the apparent variability of cortical spike trains this yields a strongly fluctuating membrane potential, such that threshold crossings are highly irregular. Here we study how a neuron could maximize its sensitivity w.r.t. a relatively small subset of excitatory input. Weak signals embedded in fluctuations is the natural realm of stochastic resonance. The neuron's response is described in a hazard-function approximation applied to an Ornstein-Uhlenbeck process.