Goto

Collaborating Authors

 avalanch


Temporal Complexity and Self-Organization in an Exponential Dense Associative Memory Model

Cafiso, Marco, Paradisi, Paolo

arXiv.org Machine Learning

Dense Associative Memory (DAM) models generalize the classical Hopfield model by incorporating n-body or exponential interactions that greatly enhance storage capacity. While the criticality of DAM models has been largely investigated, mainly within a statistical equilibrium picture, little attention has been devoted to the temporal self-organizing behavior induced by learning. In this work, we investigate the behavior of a stochastic exponential DAM (SEDAM) model through the lens of Temporal Complexity (TC), a framework that characterizes complex systems by intermittent transition events between order and disorder and by scale-free temporal statistics. Transition events associated with birth-death of neural avalanche structures are exploited for the TC analyses and compared with analogous transition events based on coincidence structures. We systematically explore how TC indicators depend on control parameters, i.e., noise intensity and memory load. Our results reveal that the SEDAM model exhibits regimes of complex intermittency characterized by nontrivial temporal correlations and scale-free behavior, indicating the spontaneous emergence of self-organizing dynamics. These regimes emerge in small intervals of noise intensity values, which, in agreement with the extended criticality concept, never shrink to a single critical point. Further, the noise intensity range needed to reach the critical region, where self-organizing behavior emerges, slightly decreases as the memory load increases. This study highlights the relevance of TC as a complementary framework for understanding learning and information processing in artificial and biological neural systems, revealing the link between the memory load and the self-organizing capacity of the network.


Tuning Universality in Deep Neural Networks

Ghavasieh, Arsham

arXiv.org Artificial Intelligence

Deep neural networks (DNNs) exhibit crackling-like avalanches whose origin lacks a mechanistic explanation. Here, I derive a stochastic theory of deep information propagation (DIP) by incorporating Central Limit Theorem (CLT)-level fluctuations. Four effective couplings $(r, h, D_1, D_2)$ characterize the dynamics, yielding a Landau description of the static exponents and a Directed Percolation (DP) structure of activity cascades. Tuning the couplings selects between avalanche dynamics generated by a Brownian Motion (BM) in a logarithmic trap and an absorbed free BM, each corresponding to a distinct universality classes. Numerical simulations confirm the theory and demonstrate that activation function design controls the collective dynamics in random DNNs.


Toward a Physics of Deep Learning and Brains

Ghavasieh, Arsham, Vila-Minana, Meritxell, Khurd, Akanksha, Beggs, John, Ortiz, Gerardo, Fortunato, Santo

arXiv.org Artificial Intelligence

Deep neural networks and brains both learn and share superficial similarities: processing nodes are likened to neurons and adjustable weights are likened to modifiable synapses. But can a unified theoretical framework be found to underlie them both? Here we show that the equations used to describe neuronal avalanches in living brains can also be applied to cascades of activity in deep neural networks. These equations are derived from non-equilibrium statistical physics and show that deep neural networks learn best when poised between absorbing and active phases. Because these networks are strongly driven by inputs, however, they do not operate at a true critical point but within a quasi-critical regime -- one that still approximately satisfies crackling noise scaling relations. By training networks with different initializations, we show that maximal susceptibility is a more reliable predictor of learning than proximity to the critical point itself. This provides a blueprint for engineering improved network performance. Finally, using finite-size scaling we identify distinct universality classes, including Barkhausen noise and directed percolation. This theoretical framework demonstrates that universal features are shared by both biological and artificial neural networks.


Monitoring snow avalanches from SAR data with deep learning

Bianchi, Filippo Maria, Grahn, Jakob

arXiv.org Artificial Intelligence

Snow avalanches present significant risks to human life and infrastructure, particularly in mountainous regions, making effective monitoring crucial. Traditional monitoring methods, such as field observations, are limited by accessibility, weather conditions, and cost. Satellite-borne Synthetic Aperture Radar (SAR) data has become an important tool for large-scale avalanche detection, as it can capture data in all weather conditions and across remote areas. However, traditional processing methods struggle with the complexity and variability of avalanches. This chapter reviews the application of deep learning for detecting and segmenting snow avalanches from SAR data. Early efforts focused on the binary classification of SAR images, while recent advances have enabled pixel-level segmentation, providing greater accuracy and spatial resolution. A case study using Sentinel-1 SAR data demonstrates the effectiveness of deep learning models for avalanche segmentation, achieving superior results over traditional methods. We also present an extension of this work, testing recent state-of-the-art segmentation architectures on an expanded dataset of over 4,500 annotated SAR images. The best-performing model among those tested was applied for large-scale avalanche detection across the whole of Norway, revealing important spatial and temporal patterns over several winter seasons.

  Country:
  Genre: Research Report > New Finding (0.46)
  Industry: Energy (0.49)

Deciphering Acoustic Emission with Machine Learning

Berta, Dénes, Katzer, Balduin, Schulz, Katrin, Ispánovity, Péter Dusán

arXiv.org Artificial Intelligence

Acoustic emission signals have been shown to accompany avalanche-like events in materials, such as dislocation avalanches in crystalline solids, collapse of voids in porous matter or domain wall movement in ferroics. The data provided by acoustic emission measurements is tremendously rich, but it is rather challenging to precisely connect it to the characteristics of the triggering avalanche. In our work we propose a machine learning based method with which one can infer microscopic details of dislocation avalanches in micropillar compression tests from merely acoustic emission data. As it is demonstrated in the paper, this approach is suitable for the prediction of the force-time response as it can provide outstanding prediction for the temporal location of avalanches and can also predict the magnitude of individual deformation events. Various descriptors (including frequency dependent and independent ones) are utilised in our machine learning approach and their importance in the prediction is analysed. The transferability of the method to other specimen sizes is also demonstrated and the possible application in more generic settings is discussed. Introduction It was shown that at the micron-scale and below crystalline materials (as well as many other heterogeneous materials) exhibit complex deformation behaviour including size-related hardening and significant sample-to-sample variation in the plastic response [1, 2, 3]. In addition, in this regime deformation becomes intermittent and constitutes of a series of random strain bursts that make the details of the deformation process unpredictable both in time and space [4, 5]. This intermittency and stochasticity originates from the sudden rearrangement events of the dislocation network, the so-called dislocation avalanches. In order to experimentally study the underlying physical process, that is, the source of the avalanche, it has to be connected to directly related proxies that are experimentally measurable with sufficient precision. This is usually a rather challenging task, since avalanches are fast and mainly occur inside the material below its surface.


The Math of the Amazing Sandpile - Issue 107: The Edge

Nautilus

One country going Communist was supposed to topple the next, and then the next, and the next. The metaphor drove much of United States foreign policy in the middle of the 20th century. But it had the wrong name. From a physical point of view, it should have been called the "sandpile theory." Real-world political phase transitions tend to happen not in neat sequences, but in sudden coordinated fits, like the Arab Spring, or the collapse of the Eastern Bloc.


Meteorologists Aim to Use AI To Get an Edge on Natural Hazards and Disasters - AI Trends

#artificialintelligence

Meteorologists are aiming to use AI to help them get an edge in early detection and disaster relief in response to natural hazards and disasters, which according to scientists have become more frequent and unpredictable due to the impact of climate change. In response, the International Telecommunication Union (ITU) together with the World Meteorological Organization (WMO) and UN Environment, have launched a Focus Group on AI for Natural Disaster Management, according to a recent account from MyITU. ITU scientists see that Al shows great potential to support data collection and monitoring, the reconstruction and forecasting of extreme events, and effective and accessible communication before and during a disaster. The ITU, founded in 1865 to facilitate international connectivity in communications networks, is today a UN specialized agency with 193 member countries and a membership of over 900 companies, universities, and international and regional organizations. The group recently held its first Focus Group on AI workshop meeting.


Using AI to better understand natural hazards and disasters

#artificialintelligence

As the realities of climate change take hold across the planet, the risks of natural hazards and disasters are becoming ever more familiar. Meteorologists, aiming to protect increasingly populous countries and communities, are tapping into artificial intelligence (AI) to get them the edge in early detection and disaster relief. This potential was in focus at a recent workshop feeding into the first meeting of the new Focus Group on AI for Natural Disaster Management. The group is open to all interested parties, supported by the International Telecommunication Union (ITU) together with the World Meteorological Organization (WMO) and UN Environment. "AI can help us tackle disasters in development work as well as standardization work. With this new Focus Group, we will explore AI's ability to analyze large datasets, refine datasets and accelerate disaster-management interventions," said Chaesub Lee, Director of the ITU Telecommunication Standardization Bureau, in opening remarks to the workshop.


Financial trading bots have fascinating similarities to people – we need to learn from them

#artificialintelligence

In 2019, the world fretted that algorithms now know us better than we know ourselves. No concept captures this better than surveillance capitalism, a term coined by American writer Shoshana Zuboff to describe a bleak new era in which the likes of Facebook and Google provide popular services while their algorithms hawk our digital traces. Surprisingly, Zuboff's concern doesn't extend to the algorithms in financial markets that have replaced many of the humans on trading floors. Automated algorithmic trading took off around the beginning of the 21st century, first in the US but soon in Europe as well. One important driver was high-frequency trading, which runs at blinding speeds, down to billionths of a second.


Snow avalanche segmentation in SAR images with Fully Convolutional Neural Networks

Bianchi, Filippo Maria, Grahn, Jakob, Eckerstorfer, Markus, Malnes, Eirik, Vickers, Hannah

arXiv.org Machine Learning

Knowledge about frequency and location of snow avalanche activity is essential for forecasting and mapping of snow avalanche hazard. Traditional field monitoring of avalanche activity has limitations, especially when surveying large and remote areas. In recent years, avalanche detection in Sentinel-1 radar satellite imagery has been developed to overcome this monitoring problem. Current state-of-the-art detection algorithms, based on radar signal processing techniques, have highly varying accuracy that is on average much lower than the accuracy of visual detections from human experts. To reduce this gap, we propose a deep learning architecture for detecting avalanches in Sentinel-1 radar images. We trained a neural network on 6345 manually labelled avalanches from 117 Sentinel-1 images, each one consisting of six channels with backscatter and topographical information. Then, we tested the best network configuration on one additional SAR image. Comparing to the manual labelling (the gold standard), we achieved an F1 score above 66%, while the state-of-the-art detection algorithm produced an F1 score of 38%. A visual interpretation of the network's results shows that it only fails to detect small avalanches, while it manages to detect some that were not labelled by the human expert.