Partially Observable Markov Decision Processes (POMDPs) are a popular formalism for sequential decision making in partially observable environments. Since solving POMDPs to optimality is a difficult task, point-based value iteration methods are widely used. These methods compute an approximate POMDP solution, and in some cases they even provide guarantees on the solution quality, but these algorithms have been designed for problems with an infinite planning horizon. In this paper we discuss why state-of-the-art point-based algorithms cannot be easily applied to finite-horizon problems that do not include discounting. Subsequently, we present a general point-based value iteration algorithm for finite-horizon problems which provides solutions with guarantees on solution quality. Furthermore, we introduce two heuristics to reduce the number of belief points considered during execution, which lowers the computational requirements. In experiments we demonstrate that the algorithm is an effective method for solving finite-horizon POMDPs.

According to the "Accenture Technology Vision for Energy 2019" report, the energy industry continues to play catch up with changing consumer expectations and the pace of digital transformation. Technology companies are larger (as measured by market capitalization) and are looking to disrupt all sectors including energy. Accenture says the energy industry will need to defend itself from these disruptors at a time when it is experiencing diminishing returns (despite widespread cost efficiencies) and increasing investor and activist scrutiny. The report argues that a combination of emerging technologies known as "DARQ"--distributed ledger (blockchain), artificial intelligence (AI), extended reality (XR) and quantum computing--will become the foundation for the industry's future. According to Accenture, tomorrow's energy industry leaders are already investing in the digital ecosystems of the future and the next wave of emerging technologies.

As AI capabilities rapidly mature, more and more chemicals and petroleum executives are determining where and how the technology fits within their organizations. Chemicals and petroleum CxOs are highly focused on three priority functional areas: information technology, information security, and innovation. These areas support the intensified focus on revenue growth and the customer as the value drivers for AI investments. AI implementation is not straightforward, however, and many companies are struggling with the transition. Yet, some businesses are achieving AI at scale successfully, and they are disproportionately outperforming financially.

The proven impact of machine learning models has pushed more investment toward their development. Still there are plenty more gains to be realized. From the first harnessing of economies of scale to the introduction of the assembly line, the search for new efficiencies has always been at the heart of manufacturing. Today, the greatest new gains come from the innovative combination of hardware and software. In particular, robotics has revolutionized manufacturing, allowing for greater output from fewer workers.

Using the CB Insights platform, we track where AI is heating up, from health to entertainment. Since 2013, over 3.6K AI startups have raised equity funding globally. The majority of these companies -- like unicorns UiPath, Automation Anywhere, and Face -- sell AI software-as-a-service. Others use AI to develop their core products, including Indigo Agriculture, which leverages machine learning to develop microbial seed treatments. Some other startups -- such as Graphcore, Habana, and Cerebras -- focus on hardware to support AI workloads.

AI running in the cloud might be the solution to electric vehicles' battery woes, if Bosch is on the right track. The Stuttgart, Germany-based company this morning announced a new service -- Battery in the Cloud -- designed to supplement vehicles' battery management systems by implementing protections to reduce cell aging. It's able to cut down on wear and tear by as much as 20%, the company claims, through continuous analysis of battery status, optimization of recharging processes, and delivery of energy conservation tips to drivers via in-car displays. The first customer is Beijing-based mobility giant DiDi Chuxing, which as of 2018 had 550 million users and tens of millions of drivers on its platform. Bosch says DiDi will equip a pilot vehicle fleet with its battery services in the city of Xiamen.

For the longest time, robots have been essential, even as developers tried to make them more human in the way they operate. Fast forward researchers at Cornell University have demystified this thought by going a notch higher and coming up with a robotic fish that tries to mimic how spiders initiate movements. A hydraulic robot that operates using the same principle of a spider is the latest development taking the world by storm. The robot in question leverages the power of pressurized fluids to move from one place to another. The robot underscores how researchers are increasingly trying to empower future robots to have more features as animals.

Deep learning has achieved remarkable success in diverse applications; however, its use in solving partial differential equations (PDEs) has emerged only recently. Here, we present an overview of physics-informed neural networks (PINNs), which embed a PDE into the loss of the neural network using automatic differentiation. The PINN algorithm is simple, and it can be applied to different types of PDEs, including integro-differential equations, fractional PDEs, and stochastic PDEs. Moreover, PINNs solve inverse problems as easily as forward problems. We propose a new residual-based adaptive refinement (RAR) method to improve the training efficiency of PINNs. For pedagogical reasons, we compare the PINN algorithm to a standard finite element method. We also present a Python library for PINNs, DeepXDE, which is designed to serve both as an education tool to be used in the classroom as well as a research tool for solving problems in computational science and engineering. DeepXDE supports complex-geometry domains based on the technique of constructive solid geometry, and enables the user code to be compact, resembling closely the mathematical formulation. We introduce the usage of DeepXDE and its customizability, and we also demonstrate the capability of PINNs and the user-friendliness of DeepXDE for five different examples. More broadly, DeepXDE contributes to the more rapid development of the emerging Scientific Machine Learning field.

What is a good exploration strategy for an agent that interacts with an environment in the absence of external rewards? Ideally, we would like to get a policy driving towards a uniform state-action visitation (highly exploring) in a minimum number of steps (fast mixing), in order to ease efficient learning of any goal-conditioned policy later on. Unfortunately, it is remarkably arduous to directly learn an optimal policy of this nature. In this paper, we propose a novel surrogate objective for learning highly exploring and fast mixing policies, which focuses on maximizing a lower bound to the entropy of the steady-state distribution induced by the policy. In particular, we introduce three novel lower bounds, that lead to as many optimization problems, that tradeoff the theoretical guarantees with computational complexity. Then, we present a model-based reinforcement learning algorithm, IDE$^{3}$AL, to learn an optimal policy according to the introduced objective. Finally, we provide an empirical evaluation of this algorithm on a set of hard-exploration tasks.

Models play an essential role in the design process of cyber-physical systems. They form the basis for simulation and analysis and help in identifying design problems as early as possible. However, the construction of models that comprise physical and digital behavior is challenging. Therefore, there is considerable interest in learning such hybrid behavior by means of machine learning which requires sufficient and representative training data covering the behavior of the physical system adequately. In this work, we exploit a combination of automata learning and model-based testing to generate sufficient training data fully automatically. Experimental results on a platooning scenario show that recurrent neural networks learned with this data achieved significantly better results compared to models learned from randomly generated data. In particular, the classification error for crash detection is reduced by a factor of five and a similar F1-score is obtained with up to three orders of magnitude fewer training samples.