Data-driven learning of partial differential equations' solution operators has recently emerged as a promising paradigm for approximating the underlying solutions. The solution operators are usually parameterized by deep learning models that are built upon problem-specific inductive biases. An example is a convolutional or a graph neural network that exploits the local grid structure where functions' values are sampled. The attention mechanism, on the other hand, provides a flexible way to implicitly exploit the patterns within inputs, and furthermore, relationship between arbitrary query locations and inputs. In this work, we present an attention-based framework for data-driven operator learning, which we term Operator Transformer (OFormer). Our framework is built upon self-attention, cross-attention, and a set of point-wise multilayer perceptrons (MLPs), and thus it makes few assumptions on the sampling pattern of the input function or query locations. We show that the proposed framework is competitive on standard benchmark problems and can flexibly be adapted to randomly sampled input.

This tutorial survey provides an overview of recent non-asymptotic advances in statistical learning theory as relevant to control and system identification. While there has been substantial progress across all areas of control, the theory is most well-developed when it comes to linear system identification and learning for the linear quadratic regulator, which are the focus of this manuscript. From a theoretical perspective, much of the labor underlying these advances has been in adapting tools from modern high-dimensional statistics and learning theory. While highly relevant to control theorists interested in integrating tools from machine learning, the foundational material has not always been easily accessible. To remedy this, we provide a self-contained presentation of the relevant material, outlining all the key ideas and the technical machinery that underpin recent results. We also present a number of open problems and future directions.

The role of artificial intelligence (AI) in material science and engineering (MSE) is becoming increasingly important as AI technology advances. The development of high-performance computing has made it possible to test deep learning (DL) models with significant parameters, providing an opportunity to overcome the limitation of traditional computational methods, such as density functional theory (DFT), in property prediction. Machine learning (ML)-based methods are faster and more accurate than DFT-based methods. Furthermore, the generative adversarial networks (GANs) have facilitated the generation of chemical compositions of inorganic materials without using crystal structure information. These developments have significantly impacted material engineering (ME) and research. Some of the latest developments in AI in ME herein are reviewed. First, the development of AI in the critical areas of ME, such as in material processing, the study of structure and material property, and measuring the performance of materials in various aspects, is discussed. Then, the significant methods of AI and their uses in MSE, such as graph neural network, generative models, transfer of learning, etc. are discussed. The use of AI to analyze the results from existing analytical instruments is also discussed. Finally, AI's advantages, disadvantages, and future in ME are discussed.

Model Predictive Control (MPC) is attracting tremendous attention in the autonomous driving task as a powerful control technique. The success of an MPC controller strongly depends on an accurate internal dynamics model. However, the static parameters, usually learned by system identification, often fail to adapt to both internal and external perturbations in real-world scenarios. In this paper, we firstly (1) reformulate the problem as a Partially Observed Markov Decision Process (POMDP) that absorbs the uncertainties into observations and maintains Markov property into hidden states; and (2) learn a recurrent policy continually adapting the parameters of the dynamics model via Recurrent Reinforcement Learning (RRL) for optimal and adaptive control; and (3) finally evaluate the proposed algorithm (referred as $\textit{MPC-RRL}$) in CARLA simulator and leading to robust behaviours under a wide range of perturbations.

In the last decade, we have witnessed a truly remarkable rise of Artificial Intelligence, Statistical and Deep learning methods (for a non exhaustive list of papers on the history of deep learning, see [1, 2, 3], and references therein). Inspired by the incredible results obtained thanks to the application of such methods to scientific problems, its adoption in the field of nuclear imaging applied to Cultural Heritage (CH) has begun (see, e.g., [4, 5, 6, 7, 8], especially the nice overview [9], and, of course, the references therein), also in the field of X-ray fluorescence Macro mapping (MA-XRF) [10, 11, 12, 13, 14, 15]. In MA-XRF, the imaging apparatus produces a data cube which, for each pixel, is formed by a spectrum containing fluorescence lines associated with the element composition of the pigment present in the pictorial layers. MA-XRF data cubes offer an ideal framework for application of unsupervised statistical learning methods [13], due to the huge number of pixel XRF histogram w.r.t. the relatively small number of employed pigment palettes. Unfortunately, in the realm of supervised statistical (deep) learning applied to CH-based analysis, the situation is flipped [14], since the data cube production is slow, obtaining a small dataset for the complexity of the various task at hand (like automatic pigment identification [14, 15], element recognition [16], and even colour association [11, 12]). This justifies the emphasis put on the creation of ad hoc synthetic MA-XRF dataset [14].

RegHEC is a registration-based hand-eye calibration technique with no need for accurate calibration rig but arbitrary available objects, applicable for both eye-in-hand and eye-to-hand cases. It tries to find the hand-eye relation which brings multi-view point clouds of arbitrary scene into simultaneous registration under a common reference frame. RegHEC first achieves initial alignment of multi-view point clouds via Bayesian optimization, where registration problem is modeled as a Gaussian process over hand-eye relation and the covariance function is modified to be compatible with distance metric in 3-D motion space SE(3), then passes the initial guess of hand-eye relation to an Anderson Accelerated ICP variant for later fine registration and accurate calibration. RegHEC has little requirement on calibration object, it is applicable with sphere, cone, cylinder and even simple plane, which can be quite challenging for correct point cloud registration and sensor motion estimation using existing methods. While suitable for most 3-D vision guided tasks, RegHEC is especially favorable for robotic 3-D reconstruction, as calibration and multi-view point clouds registration of reconstruction target are unified into a single process. Our technique is verified with extensive experiments using varieties of arbitrary objects and real hand-eye system. We release an open-source C++ implementation of RegHEC.

However, the performance of most equalizers depends on their complexity, leading to power-hungry receivers when implemented on digital hardware. Compared to conventional digital hardware, neuromorphic hardware can massively reduce energy consumption when solving the same tasks [1]. Spiking neural networks (SNNs) implemented on neuromorphic hardware mimic the human brain's behavior and promise energy-efficient, low-latency processing [2]. In [3], an SNN-based equalizer with a decision feedback structure (SNN-DFE) has been proposed for equalization and demapping based on future and currently received, and already decided symbols. For different multipath scenarios, i.e., linear channels, the SNN-DFE performs similarly to the classical decision feedback equalizer (CDFE) and artificial neural network (ANN) based equalizers. For a 4-fold pulse amplitude modulation (PAM4) transmitted over an intensity modulation / direct detection (IM/DD) link suffering from chromatic dispersion (CD) and non-linear impairments, [4] proposes an SNN that estimates the transmit symbols based on received symbols without feedback, no-feedback-SNN (NF-SNN).

Large Language Models (LLMs) have shown great potential in solving complex problems in various fields, including oil and gas engineering and other industrial engineering disciplines like factory automation, PLC programming etc. However, automatic identification of strong and weak solutions to fundamental physics equations governing several industrial processes remain a challenging task. This paper identifies the limitation of current LLM approaches, particularly ChatGPT in selected practical problems native to oil and gas engineering but not exclusively. The performance of ChatGPT in solving complex problems in oil and gas engineering is discussed and the areas where LLMs are most effective are presented.

Geophysics has witnessed success in applying deep learning to one of its core problems: full waveform inversion (FWI) to predict subsurface velocity maps from seismic data. It is treated as an image-to-image translation problem, jointly training an encoder for seismic data and a decoder for the velocity map from seismic-velocity pairs. In this paper, we report a surprising phenomenon: when training an encoder and decoder separately in their own domains via self-supervised learning, a linear relationship is observed across domains in the latent spaces. Moreover, this phenomenon connects multiple FWI datasets in an elegant manner: these datasets can share the self-learned encoder and decoder with different linear mappings. Based on these findings, we develop SimFWI, a new paradigm that includes two steps: (a) learning a seismic encoder and a velocity decoder separately by masked image modeling over multiple datasets; (b) learning a linear mapping per dataset. Experimental results show that SimFWI can achieve comparable results to a jointly trained model from the supervision of paired seismic data and velocity maps.

Deep Ensembles (DE) are a prominent approach for achieving excellent performance on key metrics such as accuracy, calibration, uncertainty estimation, and out-of-distribution detection. However, hardware limitations of real-world systems constrain to smaller ensembles and lower-capacity networks, significantly deteriorating their performance and properties. We introduce Packed-Ensembles (PE), a strategy to design and train lightweight structured ensembles by carefully modulating the dimension of their encoding space. We leverage grouped convolutions to parallelize the ensemble into a single shared backbone and forward pass to improve training and inference speeds. PE is designed to operate within the memory limits of a standard neural network. Our extensive research indicates that PE accurately preserves the properties of DE, such as diversity, and performs equally well in terms of accuracy, calibration, out-of-distribution detection, and robustness to distribution shift. We make our code available at https://github.com/ENSTA-U2IS/torch-uncertainty.