Working at Royal Dutch Shell's Deepwater division in New Orleans gives Barbara Waelde a front-row seat to how the right data can unlock crucial information for the oil giant. So when her supervisor asked her last year if she was interested in a program that could sharpen her digital and data science capabilities, Waelde, 55, jumped at the chance. Since she began her online coursework, the seven-year Shell veteran has learned Python programming, supervised learning algorithms and data modeling, among other skills. Shell began making these online courses available to U.S. employees long before COVID-19 upended daily life. And according to the oil giant, there are no plans to halt or cancel any of them, despite the fact that on March 23 it announced plans to slash operating costs by $9 billion.

Intrinsic motivation enables reinforcement learning (RL) agents to explore when rewards are very sparse, where traditional exploration heuristics such as Boltzmann or ɛ-greedy would typically fail. However, intrinsic exploration is generally handled in an ad-hoc manner, where exploration is not treated as a core objective of the learning process; this weak formulation leads to sub-optimal exploration performance. To overcome this problem, we propose a framework based on multi-objective RL where both exploration and exploitation are being optimized as separate objectives. This formulation brings the balance between exploration and exploitation at a policy level, resulting in advantages over traditional methods. This also allows for controlling exploration while learning, at no extra cost. Such strategies achieve a degree of control over agent exploration that was previously unattainable with classic or intrinsic rewards. We demonstrate scalability to continuous state-action spaces by presenting a method (EMU-Q) based on our framework, guiding exploration towards regions of higher value-function uncertainty. EMU-Q is experimentally shown to outperform classic exploration techniques and other intrinsic RL methods on a continuous control benchmark and on a robotic manipulator.

The sensitivity of heterogeneous energetic (HE) materials (propellants, explosives, and pyrotechnics) is critically dependent on their microstructure. Initiation of chemical reactions occurs at hot spots due to energy localization at sites of porosities and other defects. Emerging multi-scale predictive models of HE response to loads account for the physics at the meso-scale, i.e. at the scale of statistically representative clusters of particles and other features in the microstructure. Meso-scale physics is infused in machine-learned closure models informed by resolved meso-scale simulations. Since microstructures are stochastic, ensembles of meso-scale simulations are required to quantify hot spot ignition and growth and to develop models for microstructure-dependent energy deposition rates. We propose utilizing generative adversarial networks (GAN) to spawn ensembles of synthetic heterogeneous energetic material microstructures. The method generates qualitatively and quantitatively realistic microstructures by learning from images of HE microstructures. We show that the proposed GAN method also permits the generation of new morphologies, where the porosity distribution can be controlled and spatially manipulated. Such control paves the way for the design of novel microstructures to engineer HE materials for targeted performance in a materials-by-design framework.

The trend is to implement intelligent agents capable of analyzing available information and utilize it efficiently. This work presents a number of reinforcement learning (RL) architectures; one of them is designed for intelligent agents. The proposed architectures are called selector-actor-critic (SAC), tuner-actor-critic (TAC), and estimator-selector-actor-critic (ESAC). These architectures are improved models of a well known architecture in RL called actor-critic (AC). In AC, an actor optimizes the used policy, while a critic estimates a value function and evaluate the optimized policy by the actor. SAC is an architecture equipped with an actor, a critic, and a selector. The selector determines the most promising action at the current state based on the last estimate from the critic. TAC consists of a tuner, a model-learner, an actor, and a critic. After receiving the approximated value of the current state-action pair from the critic and the learned model from the model-learner, the tuner uses the Bellman equation to tune the value of the current state-action pair. ESAC is proposed to implement intelligent agents based on two ideas, which are lookahead and intuition. Lookahead appears in estimating the values of the available actions at the next state, while the intuition appears in maximizing the probability of selecting the most promising action. The newly added elements are an underlying model learner, an estimator, and a selector. The model learner is used to approximate the underlying model. The estimator uses the approximated value function, the learned underlying model, and the Bellman equation to estimate the values of all actions at the next state. The selector is used to determine the most promising action at the next state, which will be used by the actor to optimize the used policy. Finally, the results show the superiority of ESAC compared with the other architectures.

Thanks to the advent of the latest innovations in Artificial Intelligence (AI) and machine learning (ML), smart cities -- with a specific focus on the utilities sector -- are undergoing unprecedented changes. The Capgemini Research Institute estimated that, together with the energy sector, the utility vertical can save between $237 billion to $813 billion USD from intelligent automation at scale. Utility companies have been experimenting with AI use cases such as predictive maintenance, yield optimization, and demand/load forecasting. In 2019, more than half of energy and utilities organizations have deployed at least one practical implementation of AI technology, reaping its consistent benefits. Even the public seems eager to enjoy the positive innovations brought forward by the AI transformation.

Researchers at the Center for Nanoscale Materials (CNM), a U.S. Department of Energy (DOE) Office of Science User Facility located at the DOE's Argonne National Laboratory, have invented a machine-learning based algorithm for quantitatively characterizing, in three dimensions, materials with features as small as nanometers. Researchers can apply this pivotal discovery to the analysis of most structural materials of interest to industry. "What makes our algorithm unique is that if you start with a material for which you know essentially nothing about the microstructure, it will, within seconds, tell the user the exact microstructure in all three dimensions," said Subramanian Sankaranarayanan, group leader of the CNM theory and modeling group and an associate professor in the Department of Mechanical and Industrial Engineering at the University of Illinois at Chicago. "For example, with data analyzed by our 3D tool," said Henry Chan, CNM postdoctoral researcher and lead author of the study, "users can detect faults and cracks and potentially predict the lifetimes under different stresses and strains for all kinds of structural materials." Most structural materials are polycrystalline, meaning a sample used for purposes of analysis can contain millions of grains.

Artificial Intelligence (AI) in the oil and gas industry stands to reach US$2.85 billion by 2022. Because data is never special. Oil rigs may generate somewhere around 50 terabytes a year, but that kind of big data needs to be applicable to be useful and, unfortunately, humans do a terrible job of classifying things into datasets. Indeed, a good scenario will see 10% of the resulting datasets actually be beneficial. Most competing firms are also known to have access to the same datasets.

Model Predictive Control has been recently proposed as policy approximation for Reinforcement Learning, offering a path towards safe and explainable Reinforcement Learning. This approach has been investigated for Q-learning and actor-critic methods, both in the context of nominal Economic MPC and Robust (N)MPC, showing very promising results. In that context, actor-critic methods seem to be the most reliable approach. Many applications include a mixture of continuous and integer inputs, for which the classical actor-critic methods need to be adapted. In this paper, we present a policy approximation based on mixed-integer MPC schemes, and propose a computationally inexpensive technique to generate exploration in the mixed-integer input space that ensures a satisfaction of the constraints. We then propose a simple compatible advantage function approximation for the proposed policy, that allows one to build the gradient of the mixed-integer MPC-based policy.

Recently, various relations and criteria have been presented to establish a proper relationship between control systems and control the Global Positioning System (GPS)-intelligent buoy system. Given the importance of controlling the position of buoys and the construction of intelligent systems, in this paper, dynamic system modeling is applied to position marine buoys through the improved neural network with a backstepping technique. This study aims at developing a novel controller based on an adaptive fuzzy neural network to optimally track the dynamically positioned vehicle on the water with unavailable velocities and unidentified control parameters. In order to model the network with the proposed technique, uncertainties and the unwanted disturbances are studied in the neural network. The presented study aims at developing a neural controlling which applies the vectorial back-stepping technique to the surface ships, which have been dynamically positioned with undetermined disturbances and ambivalences. Moreover, the objective function is to minimize the output error for the neural network (NN) based on the closed-loop system. The most important feature of the proposed model for the positioning buoys is its independence from comparative knowledge or information on the dynamics and the unwanted disturbances of ships. The numerical and obtained consequences demonstrate that the control system can adjust the routes and the position of the buoys to the desired objective with relatively few position errors.

This paper presents a generalized flexible Hybrid Cable-Driven Robot (HCDR). For the proposed HCDR, the derivation of the equations of motion and proof provide a very effective way to find items for generalized system modeling. The proposed dynamic modeling approach avoids the drawback of traditional methods and can be easily extended to other types of hybrid robots, such as a robot arm mounted on an aircraft platform. Additionally, another goal of this paper is to develop integrated control systems to reduce vibrations and improve the accuracy and performance of the HCDR. To achieve this goal, redundancy resolution, stiffness optimization, and control strategies are studied. The proposed optimization problem and algorithm address the limitations of existing stiffness optimization approaches. Three types of control architecture are proposed, and their performances (i.e., reducing undesirable vibrations and trajectory tracking errors, especially for the end-effector) are evaluated using several well-designed case studies. Results show that the fully integrated control strategy can improve the tracking performance of the end-effector significantly.