Condition-based monitoring (CBM) has been widely utilised in the wind industry for monitoring operational inconsistencies and failures in turbines, with techniques ranging from signal processing and vibration analysis to artificial intelligence (AI) models using Supervisory Control & Acquisition (SCADA) data. However, existing studies do not present a concrete basis to facilitate explainable decision support in operations and maintenance (O&M), particularly for automated decision support through recommendation of appropriate maintenance action reports corresponding to failures predicted by CBM techniques. Knowledge graph databases (KGs) model a collection of domain-specific information and have played an intrinsic role for real-world decision support in domains such as healthcare and finance, but have seen very limited attention in the wind industry. We propose XAI4Wind, a multimodal knowledge graph for explainable decision support in real-world operational turbines and demonstrate through experiments several use-cases of the proposed KG towards O&M planning through interactive query and reasoning and providing novel insights using graph data science algorithms. The proposed KG combines multimodal knowledge like SCADA parameters and alarms with natural language maintenance actions, images etc. By integrating our KG with an Explainable AI model for anomaly prediction, we show that it can provide effective human-intelligible O&M strategies for predicted operational inconsistencies in various turbine sub-components. This can help instil better trust and confidence in conventionally black-box AI models. We make our KG publicly available and envisage that it can serve as the building ground for providing autonomous decision support in the wind industry.

Belief propagation is an algorithm that is known from statistical physics and computer science. It provides an efficient way of calculating marginals that involve large sums of products which are efficiently rearranged into nested products of sums to approximate the marginals. It allows a reliable estimation of the state and its variance of power grids that is needed for the control and forecast of power grid management. At prototypical examples of IEEE-grids we show that belief propagation not only scales linearly with the grid size for the state estimation itself, but also facilitates and accelerates the retrieval of missing data and allows an optimized positioning of measurement units. Based on belief propagation, we give a criterion for how to assess whether other algorithms, using only local information, are adequate for state estimation for a given grid. We also demonstrate how belief propagation can be utilized for coarse-graining power grids towards representations that reduce the computational effort when the coarse-grained version is integrated into a larger grid. It provides a criterion for partitioning power grids into areas in order to minimize the error of flow estimates between different areas.

Two years ago, NASA's InSight spacecraft alighted on the surface of Mars, aiming to glean clues to the planet's interior from the shaking of distant earthquakes and deep heat leaking from its soil. Mars, it turned out, had other ideas. Its sticky soil has thwarted InSight's heat probe, and in recent months howling winds have deafened its sensitive seismometers. Most mysteriously, the planet hasn't been rattled by the large marsquakes that could vividly illuminate its depths. Despite these hurdles, a precious clutch of small-but-clear quakes has enabled the InSight team to see hints of boundaries in the rock, tens and hundreds of kilometers below. They are clues to the planet's formation billions of years ago, when it was a hot ball of magma and heavier elements like iron sank to form a core, while lighter rocks rose up out of the mantle to form a capping lid of crust. The results, some debuting this month at an online meeting of the American Geophysical Union (AGU), show that the planet's crust is surprisingly thin, its mantle cooler than expected, and its large iron core still molten. The findings suggest that in its infancy, Mars efficiently shed heat—perhaps through a pattern of upwelling mantle rock and subducting crust similar to plate tectonics on Earth. “This may be evidence for a far more dynamic crust formation in Mars's early days,” says Stephen Mojzsis, a planetary scientist at the University of Colorado, Boulder, who is unaffiliated with the mission. The evidence has been hard won. Early in the mission, winds were quiet enough for InSight's seismometers, housed in a small dome placed on the surface, to hear a multitude of small quakes—nearly 500 in total. But since June, winds have shaken the surface strongly enough to smother all but a handful of new quakes. Yet frustratingly, the winds have not been strong enough to sweep away dust that is darkening the craft's solar panels and foreshadowing the mission's end sometime in the next few years. The seismometers are still running nonstop, but power constraints have forced the team to turn off a weather station when using the lander's robotic arm. “We are starting to feel the effects,” says Bruce Banerdt, InSight's principal investigator and a geophysicist at NASA's Jet Propulsion Laboratory. Meanwhile, the heat probe, about the length of a paper towel tube, is stuck in soil that compacted instead of crumbling as the rod tried to delve in. Mission engineers have used the robotic arm to push the probe down and scrape dirt on top. In the next month or two, they'll try once more to get the probe to burrow in, Banerdt says. “If that doesn't work, we'll call it a day and accept disappointment.” Perhaps the biggest disappointment is the lack of a marsquake larger than magnitude 4.5. The seismic waves of a large quake travel more deeply, reflecting off the core and mantle boundaries and even circling the planet on its surface. The multiple echoes of a large quake can enable just a single seismic station like InSight's to locate the quake's source. But above magnitude 4, Mars has been curiously silent—an apparent violation of the scaling laws that apply on Earth and the Moon, where 100 magnitude 3 events correspond to 10 magnitude 4 quakes, and so on. “That is a bit weird,” says Simon Stähler, a seismologist on the team from ETH Zurich. It could simply be that Mars's faults aren't big enough to sustain big strikes, or that its crust isn't brittle enough. But two moderate quakes, at magnitude 3.7 and 3.3, have been treasure troves for the mission. Traced to Cerberus Fossae, deep fissures in the crust 1600 kilometers east of the landing site that were suspected of being seismically active, the quakes sent a one-two punch of compressive pressure (P) waves, followed by sidewinding shear (S) waves, barreling toward the lander. Some of the waves were confined to the crust; others reflected off the top of the mantle. Offsets in the travel times of the P and S waves hint at the thickness of the crust and suggest distinct layers within it, Brigitte Knapmeyer-Endrun, a seismologist at the University of Cologne, said in an AGU presentation. The top layer may reflect material ground up in the planet's first billion years, a period of intense asteroid bombardment, says Steven Hauck, a planetary scientist at Case Western Reserve University. At 20 or 37 kilometers thick, depending on whether the reflections accurately trace the top of the mantle, the martian crust appears to be thinner than Earth's continental crust—a surprise. Researchers had thought that Mars, a smaller planet with less internal heat, would have built up a thicker crust, with heat escaping through limited conduction and bouts of volcanism. (Though Mars is volcanically dead today, giant volcanoes dot its surface.) A thin crust, however, might mean Mars was losing heat efficiently, recycling its early crust, rather than just building it up, perhaps through a rudimentary form of plate tectonics, Mojzsis says. A handful of distant quakes, originating some 4000 kilometers away, provided a further clue. Those waves traveled deep through the mantle and interacted with the mantle transition zone, a layer where pressure transforms the mineral olivine into wadsleyite. By analyzing the travel time of waves that passed above, below, and through the transition zone, the team located its depth—and found it shallower than expected, an indication of a cooler mantle. For the mantle to be this cool today suggests that convection—the swirling motions that, on Earth, drive tectonic plates and carry heat from the mantle to the surface—might have operated early on, says Quancheng Huang, a Ph.D. student at the University of Maryland, College Park, who presented some of the results at the AGU meeting. “Plate tectonics is a very effective way of cooling a planet.” A third science experiment aboard InSight probes deeper still, using tiny Doppler shifts in radio broadcasts sent from Earth to receivers on the probe to detect slight wobbles in the planet's spin. The size and consistency of the planet's iron core affect the wobbles, much as raw eggs spin differently from cooked ones. “We've had something like 350 hours of tracking,” says Véronique Dehant, a geophysicist at the Royal Observatory of Belgium. The preliminary results confirm that the core is liquid, with a radius compatible with previous estimates made by spacecraft measuring tiny variations in the planet's gravity, Dehant reports in her AGU poster. Those gravity estimates have found a core with a radius of about 1800 kilometers—taking up more than half the planet's diameter. Rebecca Fischer, a mineral physicist and modeler at Harvard University, isn't surprised at the signs of a liquid core. “It would be a pretty big surprise if it weren't,” she says. Sulfur and other elements mixed with the iron should help it to remain molten while cool, much as salt prevents icing. On Earth, convective motions in the molten outer core drive the magnetic dynamo. But on Mars, those motions seem to have stopped long ago—and without a magnetic field, the planet's atmosphere was vulnerable to the Sun's cosmic rays and leached water to space. Banerdt hopes to sharpen this fuzzy picture of the planet's interior, and he thinks calmer winds will soon make that possible. After two Earth years, the probe's first martian year is ending, and the quiet of the mission's first months is returning. “We're looking forward to another whole pile of event detections,” Banerdt says. And though the planet has not cooperated so far, perhaps the Big One is poised to strike Mars like a gong—a reverberation that would at last make all clear.

Reinforcement learning holds tremendous promise in accelerator controls. The primary goal of this paper is to show how this approach can be utilised on an operational level on accelerator physics problems. Despite the success of model-free reinforcement learning in several domains, sample-efficiency still is a bottle-neck, which might be encompassed by model-based methods. We compare well-suited purely model-based to model-free reinforcement learning applied to the intensity optimisation on the FERMI FEL system. We find that the model-based approach demonstrates higher representational power and sample-efficiency, while the asymptotic performance of the model-free method is slightly superior. The model-based algorithm is implemented in a DYNA-style using an uncertainty aware model, and the model-free algorithm is based on tailored deep Q-learning. In both cases, the algorithms were implemented in a way, which presents increased noise robustness as omnipresent in accelerator control problems. Code is released in https://github.com/MathPhysSim/FERMI_RL_Paper.

As a simple and efficient optimization method in deep learning, stochastic gradient descent (SGD) has attracted tremendous attention. In the vanishing learning rate regime, SGD is now relatively well understood, and the majority of theoretical approaches to SGD set their assumptions in the continuous-time limit. However, the continuous-time predictions are unlikely to reflect the experimental observations well because the practice often runs in the large learning rate regime, where the training is faster and the generalization of models are often better. In this paper, we propose to study the basic properties of SGD and its variants in the non-vanishing learning rate regime. The focus is on deriving exactly solvable results and relating them to experimental observations. The main contributions of this work are to derive the stable distribution for discrete-time SGD in a quadratic loss function with and without momentum. Examples of applications of the proposed theory considered in this work include the approximation error of variants of SGD, the effect of mini-batch noise, the escape rate from a sharp minimum, and and the stationary distribution of a few second order methods.

Bayesian optimisation presents a sample-efficient methodology for global optimisation. Within this framework, a crucial performance-determining subroutine is the maximisation of the acquisition function, a task complicated by the fact that acquisition functions tend to be non-convex and thus nontrivial to optimise. In this paper, we undertake a comprehensive empirical study of approaches to maximise the acquisition function. Additionally, by deriving novel, yet mathematically equivalent, compositional forms for popular acquisition functions, we recast the maximisation task as a compositional optimisation problem, allowing us to benefit from the extensive literature in this field. We highlight the empirical advantages of the compositional approach to acquisition function maximisation across 3958 individual experiments comprising synthetic optimisation tasks as well as tasks from Bayesmark. Given the generality of the acquisition function maximisation subroutine, we posit that the adoption of compositional optimisers has the potential to yield performance improvements across all domains in which Bayesian optimisation is currently being applied.

We formally study how Ensemble of deep learning models can improve test accuracy, and how the superior performance of ensemble can be distilled into a single model using Knowledge Distillation. We consider the challenging case where the ensemble is simply an average of the outputs of a few independently trained neural networks with the SAME architecture, trained using the SAME algorithm on the SAME data set, and they only differ by the random seeds used in the initialization. We empirically show that ensemble/knowledge distillation in deep learning works very differently from traditional learning theory, especially differently from ensemble of random feature mappings or the neural-tangent-kernel feature mappings, and is potentially out of the scope of existing theorems. Thus, to properly understand ensemble and knowledge distillation in deep learning, we develop a theory showing that when data has a structure we refer to as "multi-view", then ensemble of independently trained neural networks can provably improve test accuracy, and such superior test accuracy can also be provably distilled into a single model by training a single model to match the output of the ensemble instead of the true label. Our result sheds light on how ensemble works in deep learning in a way that is completely different from traditional theorems, and how the "dark knowledge" is hidden in the outputs of the ensemble -- that can be used in knowledge distillation -- comparing to the true data labels. In the end, we prove that self-distillation can also be viewed as implicitly combining ensemble and knowledge distillation to improve test accuracy.

Physics-informed neural networks (PINNs) are demonstrating remarkable promise in integrating physical models with gappy and noisy observational data, but they still struggle in cases where the target functions to be approximated exhibit high-frequency or multi-scale features. In this work we investigate this limitation through the lens of Neural Tangent Kernel (NTK) theory and elucidate how PINNs are biased towards learning functions along the dominant eigen-directions of their limiting NTK. Using this observation, we construct novel architectures that employ spatio-temporal and multi-scale random Fourier features, and justify how such coordinate embedding layers can lead to robust and accurate PINN models. Numerical examples are presented for several challenging cases where conventional PINN models fail, including wave propagation and reaction-diffusion dynamics, illustrating how the proposed methods can be used to effectively tackle both forward and inverse problems involving partial differential equations with multi-scale behavior. All code an data accompanying this manuscript will be made publicly available at \url{https://github.com/PredictiveIntelligenceLab/MultiscalePINNs}.

Extreme precipitation events, such as violent rainfall and hail storms, routinely ravage economies and livelihoods around the developing world. Climate change further aggravates this issue. Data-driven deep learning approaches could widen the access to accurate multi-day forecasts, to mitigate against such events. However, there is currently no benchmark dataset dedicated to the study of global precipitation forecasts. In this paper, we introduce \textbf{RainBench}, a new multi-modal benchmark dataset for data-driven precipitation forecasting. It includes simulated satellite data, a selection of relevant meteorological data from the ERA5 reanalysis product, and IMERG precipitation data. We also release \textbf{PyRain}, a library to process large precipitation datasets efficiently. We present an extensive analysis of our novel dataset and establish baseline results for two benchmark medium-range precipitation forecasting tasks. Finally, we discuss existing data-driven weather forecasting methodologies and suggest future research avenues.

Achieving efficiency gains in Chinese district heating networks, thereby reducing their carbon footprint, requires new optimal control methods going beyond current industry tools. Focusing on the secondary network, we propose a data-driven deep reinforcement learning (DRL) approach to address this task. We build a recurrent neural network, trained on simulated data, to predict the indoor temperatures. This model is then used to train two DRL agents, with or without expert guidance, for the optimal control of the supply water temperature. Our tests in a multi-apartment setting show that both agents can ensure a higher thermal comfort and at the same time a smaller energy cost, compared to an optimized baseline strategy.