Reinforcement learning has been successfully used to solve difficult tasks in complex unknown environments. However, these methods typically do not provide any safety guarantees during the learning process. This is particularly problematic, since reinforcement learning agent actively explore their environment. This prevents their use in safety-critical, real-world applications. In this paper, we present a learning-based model predictive control scheme that provides high-probability safety guarantees throughout the learning process. Based on a reliable statistical model, we construct provably accurate confidence intervals on predicted trajectories. Unlike previous approaches, we allow for input-dependent uncertainties. Based on these reliable predictions, we guarantee that trajectories satisfy safety constraints. Moreover, we use a terminal set constraint to recursively guarantee the existence of safe control actions at every iteration. We evaluate the resulting algorithm to safely explore the dynamics of an inverted pendulum and to solve a reinforcement learning task on a cart-pole system with safety constraints.

In this work we present a novel approach for transfer-guided exploration in reinforcement learning that is inspired by the human tendency to leverage experiences from similar encounters in the past while navigating a new task. Given an optimal policy in a related task-environment, we show that its bisimulation distance from the current task-environment gives a lower bound on the optimal advantage of state-action pairs in the current task-environment. Transfer-guided Exploration (ExTra) samples actions from a Softmax distribution over these lower bounds. In this way, actions with potentially higher optimum advantage are sampled more frequently. In our experiments on gridworld environments, we demonstrate that given access to an optimal policy in a related task-environment, ExTra can outperform popular domain-specific exploration strategies viz. epsilon greedy, Model-Based Interval Estimation - Exploration Based (MBIE-EB), Pursuit and Boltzmann in terms of sample complexity and rate of convergence. We further show that ExTra is robust to choices of source task and shows a graceful degradation of performance as the dissimilarity of the source task increases. We also demonstrate that ExTra, when used alongside traditional exploration algorithms, improves their rate of convergence. Thus it is capable of complimenting the efficacy of traditional exploration algorithms.

Characterization of a non-Gaussian channelized conductivity field in subsurface flow and transport modeling through inverse modeling usually leads to a high-dimensional inverse problem and requires repeated evaluations of the forward model. In this study, we develop a convolutional adversarial autoencoder (CAAE) network to parameterize the high-dimensional non-Gaussian conductivity fields using a low-dimensional latent representation and a deep residual dense convolutional network (DRDCN) to efficiently construct a surrogate model for the forward model. The two networks are both based on a multilevel residual learning architecture called residual-in-residual dense block. The multilevel residual learning strategy and the dense connection structure in the dense block ease the training of deep networks, enabling us to efficiently build deeper networks that have an essentially increased capacity for approximating mappings of very high-complexity. The CCAE and DRDCN networks are incorporated into an iterative local updating ensemble smoother to formulate an inversion framework. The integrated method is demonstrated using a synthetic solute transport model. Results indicate that CAAE is a robust parameterization method for the channelized conductivity fields with Gaussian conductivities within each facies. The DRDCN network is able to obtain an accurate surrogate model of the forward model with high-dimensional and highly-complex concentration fields using relatively limited training data. The CAAE paramterization approach and the DRDCN surrogate method together significantly reduce the number of forward model runs required to achieve accurate inversion results.

Artificial neural network (ANN) provides superior accuracy for nonlinear alternating current (AC) state estimation (SE) in smart grid over traditional methods. However, research has discovered that ANN could be easily fooled by adversarial examples. In this paper, we initiate a new study of adversarial false data injection (FDI) attack against AC SE with ANN: by injecting a deliberate attack vector into measurements, the attacker can degrade the accuracy of ANN SE while remaining undetected. We propose a population-based algorithm and a gradient-based algorithm to generate attack vectors. The performance of these algorithms is evaluated through simulations on IEEE 9-bus, 14-bus and 30-bus systems under various attack scenarios. Simulation results show that DE is more effective than SLSQP on all simulation cases. The attack examples generated by DE algorithm successfully degrade the ANN SE accuracy with high probability.

Dempster-Shafer evidence theory has been widely used in various fields of applications, because of the flexibility and effectiveness in modeling uncertainties without prior information. However, the existing evidence theory is insufficient to consider the situations where it has no capability to express the fluctuations of data at a given phase of time during their execution, and the uncertainty and imprecision which are inevitably involved in the data occur concurrently with changes to the phase or periodicity of the data. In this paper, therefore, a generalized Dempster-Shafer evidence theory is proposed. To be specific, a mass function in the generalized Dempster-Shafer evidence theory is modeled by a complex number, called as a complex basic belief assignment, which has more powerful ability to express uncertain information. Based on that, a generalized Dempster's combination rule is exploited. In contrast to the classical Dempster's combination rule, the condition in terms of the conflict coefficient between the evidences K<1 is released in the generalized Dempster's combination rule. Hence, it is more general and applicable than the classical Dempster's combination rule. When the complex mass function is degenerated from complex numbers to real numbers, the generalized Dempster's combination rule degenerates to the classical evidence theory under the condition that the conflict coefficient between the evidences K is less than 1. In a word, this generalized Dempster-Shafer evidence theory provides a promising way to model and handle more uncertain information.

Baker Hughes, a GE Company (BHGE) has entered into a joint venture with Artificial Intelligence (AI) software provider C3.ai to accelerate the digital transformation in the oil and gas industry. C3.ai's AI suite and applications will be deployed into BHGE's business and leverage the oilfield services company's existing digital portfolio. "The oil and gas industry is rapidly evolving, and digital technology is critical to achieving increased levels of productivity, efficiency and safety for ourselves and for our customers," BHGE CEO Lorenzo Simonelli said in a company statement. "This agreement is a mutual recognition of the technology leadership we each bring to the table and a willingness to work in new ways … integrating our strong digital capabilities and oil and gas industry expertise with C3.ai's unique AI solutions, we will accelerate the overall digital transformation of this industry." Under the terms of the joint venture agreement, BHGE will take a minority equity position in C3.ai and will have a seat on C3.ai's board of directors.

We expect the landscape to be an integrated edge-to-core-to-cloud solution enabling what today is called IoT, Big Data, Fast Data and AI. Each time a promising new technology emerges, we seem to go through a period where it is proposed to be the solution to everything--until we reconcile how that technology fits into the bigger picture. Such is the case with artificial intelligence (AI). Clearly the advancements in deep learning will create new classes of solutions but rather than being a standalone solution, we are just now beginning to see how it fits into our IT landscape. AI emerges at a time when several other shifts in analytics technology are occurring.

May Masoud is a Solution Specialist at SAS Canada, as part of the Data Sciences team. Leveraging her analytics background, she helps businesses visualize the potential of their data, and surface insights using modern data mining and machine learning techniques. With a Master of Business Analytics following a Bachelor in Statistics & Economics, May aims to create value at every step of the analytics lifecycle: data discovery, model build, model deployment, and business strategy. She has touched the analytics landscape in a variety of industries, whether it is oil production models for the energy sector or solving churn problems in the telecom industry. May's aim is to ubiquitize self-serve analytics and enable citizen data scientists.

We introduce a method to design lightweight shell objects that are structurally robust under the external forces they may experience during use. Given an input 3D model and a general description of the external forces, our algorithm generates a structurally-sound minimum weight shell object. Our approach works by altering the local shell thickness repeatedly based on the stresses that develop inside the object. A key issue in shell design is that large thickness values might result in self-intersections on the inner boundary creating a significant computational challenge during optimization. To address this, we propose a shape parametrization based on the solution to the Laplace's equation that guarantees smooth and intersection-free shell boundaries. Combined with our gradient-free optimization algorithm, our method provides a practical solution to the structural design of hollow objects with a single inner cavity. We demonstrate our method on a variety of problems with arbitrary 3D models under complex force configurations and validate its performance with physical experiments.

Model-based reinforcement learning has the potential to be more sample efficient than model-free approaches. However, existing model-based methods are vulnerable to model bias, which leads to poor generalization and asymptotic performance compared to model-free counterparts. In addition, they are typically based on the model predictive control (MPC) framework, which not only is computationally inefficient at decision time but also does not enable policy transfer due to the lack of an explicit policy representation. In this paper, we propose a novel uncertainty-aware model-based policy optimization framework which solves those issues. In this framework, the agent simultaneously learns an uncertainty-aware dynamics model and optimizes the policy according to these learned models. In the optimization step, the policy gradient is computed by automatic differentiation through the models. With respect to sample efficiency alone, our approach shows promising results on challenging continuous control benchmarks with competitive asymptotic performance and significantly lower sample complexity than state-of-the-art baselines.