Following the remarkable success of the AlphaGO series, 2019 was a booming year that witnessed significant advances in multi-agent reinforcement learning (MARL) techniques. MARL corresponds to the learning problem in a multi-agent system in which multiple agents learn simultaneously. MARL is an interdisciplinary domain with a long history that includes game theory, machine learning, stochastic control, psychology, and optimisation. Although MARL has achieved considerable empirical success in solving real-world games, there is a lack of a self-contained overview in the literature that elaborates the game theoretical foundations of modern MARL methods and summarises the recent advances. In fact, the majority of existing surveys are outdated and do not fully cover the recent developments since 2010. In this work, we provide a monograph on MARL that covers both the fundamentals and the latest developments in the research frontier. The goal of our monograph is to provide a self-contained assessment of the current state-of-the-art MARL techniques from a game theoretical perspective. We expect this work to serve as a stepping stone for both new researchers who are about to enter this fast-growing domain and existing domain experts who want to obtain a panoramic view and identify new directions based on recent advances.

AI Has Cracked a Key Mathematical Puzzle for Understanding Our World Karen Hao MIT Technology Review "Partial differential equations can describe everything from planetary motion to plate tectonics, but they're notoriously hard to solve. Physicists 3D Print a Boat That Could Sail Down a Human Hair John Biggs Gizmodo "Researchers at Leiden University have 3D printed the smallest boat in the world: a 30-micrometer copy of Benchy the tug boat, a well-known 3D printer test object. This boat is so small, it could float down the interior of a human hair. The 3D-printed boat is part of an exploration of microswimmers, microscopic organisms or objects that can move through liquids." Record-Smashing Hybrid Drone Stays Airborne for a Crazy 10 Hours, 14 Minutes Luke Dormehl Digital Trends "i'HYBRiX is an innovation, inspired by hybrid cars, that combines the best of both technologies,' a spokesperson for Quaternium told Digital Trends, referring to the drone's clever gasoline and battery-electric hybrid power system.

It is known that changes in the mechanical properties of tissues are associated with the onset and progression of certain diseases. Ultrasound elastography is a technique to characterize tissue stiffness using ultrasound imaging either by measuring tissue strain using quasi-static elastography or natural organ pulsation elastography, or by tracing a propagated shear wave induced by a source or a natural vibration using dynamic elastography. In recent years, deep learning has begun to emerge in ultrasound elastography research. In this review, several common deep learning frameworks in the computer vision community, such as multilayer perceptron, convolutional neural network, and recurrent neural network are described. Then, recent advances in ultrasound elastography using such deep learning techniques are revisited in terms of algorithm development and clinical diagnosis. Finally, the current challenges and future developments of deep learning in ultrasound elastography are prospected.

Long short-term memory (LSTM) The long short-term memory (LSTM) is a special kind of recurrent neural network (Gers et al., 1999; Hochreiter & Schmidhuber, 1997), and is capable of processing sequential data with correlations between points that are far apart. On the one hand, similar to the standard recurrent neural network, the LSTM has a self-looped structure that allows the result of the previous step to participate in the calculation of the subsequent step. On the other hand, the LSTM possesses four interaction layers in its neurons, which makes it able to forget useless information and learn correlations between data points that are far away from each other in sequence. The LSTM is the state-of-the-art model for well log generation in previous studies (Zhang et al., 2018). This agrees well with the perspective of geoscience, since the well logs reflect a formation condition, which possesses internal continuity (spatial dependency). The sequential information in reservoirs is critical for well logs generation. Therefore, the LSTM constitutes the ideal foundation for building a new model for this type of geoscience problem.

For the last few years, the name AI is knocking us every day as a part of our life. A smartphone in your hand is also equipped with AI to enhance your experience. Even a selfie is also processed with AI to improve the quality of the image. Big data and AI together are impacting our lives by using our smallest details with our concern to provide us better results. We are being used to such small yet useful applications of AI but there are few applications we may have never imagined to be true this soon.

The final part of my Life series is still in the works but I need to interrupt that series with some exciting news. I will likely do a whole series on Bean Machine later on this autumn, but for today let me just give you the brief overview should you not want to go through the paper. As the paper's title says, Bean Machine is a Probabilistic Programming Language (PPL). For a detailed introduction to PPLs you should read my "Fixing Random" series, where I show how we could greatly improve support for analysis of randomness in .NET by both adding types to the base class library and by adding language features to a language like C#. If you don't want to read that 40 post introduction, here's the TLDR.

To leverage faster AI experimentation with complete GPU utilization, leading cloud data service provider NetApp is joining hands with reputed virtual AI infrastructure Run:AI. This collaboration will be beneficial for both of the companies as it will allow multiple AI experiments to run simultaneously with faster access to data and better use of unlimited computing resources. Run:AI automates resource allocation enabling full GPU utilization. With the help of NetApp ONTAP AI proven architecture, each experiment is allowed to run at maximum speed through the elimination of data pipeline bottlenecks. Overall, the collaboration of NetApp and Run:AI allows teams to gain the double benefit of full resource utilization and faster experiments for the scaling of AI.

Boston Dynamics' Spot robot is expanding to its resume every day, and the quadruped can add nuclear power plant exploration and radiation monitoring to the list. Engineers from the University of Bristol recently tested Spot around the Exclusion Zone territory of the Chernobyl Nuclear Power Plant. The Exclusion Zone covers approximately a 1,000-square-mile area in Ukraine surrounding the Chernobyl Nuclear Power Plant, where radioactive contamination is highest and public access and inhabitation are restricted. According to the State Agency for Exclusion Zone Management, this is the first time Spot has been tested there. Spot helped create a 3D map of the distribution of nuclear radiation around the Chernobyl Nuclear Plant.

The success of deep learning, a brain-inspired form of AI, has sparked interest in understanding how the brain could similarly learn across multiple layers of neurons. However, the majority of biologically-plausible learning algorithms have not yet reached the performance of backpropagation (BP), nor are they built on strong theoretical foundations. Here, we analyze target propagation (TP), a popular but not yet fully understood alternative to BP, from the standpoint of mathematical optimization. Our theory shows that TP is closely related to Gauss-Newton optimization and thus substantially differs from BP. Furthermore, our analysis reveals a fundamental limitation of difference target propagation (DTP), a well-known variant of TP, in the realistic scenario of non-invertible neural networks. We provide a first solution to this problem through a novel reconstruction loss that improves feedback weight training, while simultaneously introducing architectural flexibility by allowing for direct feedback connections from the output to each hidden layer. Our theory is corroborated by experimental results that show significant improvements in performance and in the alignment of forward weight updates with loss gradients, compared to DTP.

Wasserstein distributionally robust optimization (DRO) aims to find robust and generalizable solutions by hedging against data perturbations in Wasserstein distance. Despite its recent empirical success in operations research and machine learning, existing performance guarantees for generic loss functions are either overly conservative due to the curse of dimensionality, or plausible only in large sample asymptotics. In this paper, we develop a non-asymptotic framework for analyzing the out-of-sample performance for Wasserstein robust learning and the generalization bound for its related Lipschitz and gradient regularization problems. To the best of our knowledge, this gives the first finite-sample guarantee for generic Wasserstein DRO problems without suffering from the curse of dimensionality. Our results highlight the bias-variation trade-off intrinsic in the Wasserstein DRO, which balances between the empirical mean of the loss and the variation of the loss, measured by the Lipschitz norm or the gradient norm of the loss. Our analysis is based on two novel methodological developments that are of independent interest: 1) a new concentration inequality controlling the decay rate of large deviation probabilities by the variation of the loss and, 2) a localized Rademacher complexity theory based on the variation of the loss.