We've found crowdsourcing (i.e., the gathering of ideas, thoughts, and concepts from a widespread variety of interested individuals) to be a very effective tool in enabling us to diversify our thoughts and challenge our assumptions. Dr. Buras' post takes the results from one such crowdsourcing exercise and extrapolates three future urban scenarios. Given The Army Vision's clarion call to "Focus training on high-intensity conflict, with emphasis on operating in dense urban terrain," our readers would do well to consider how the Army would operate in each of Dr. Buras' posited future scenarios…] The challenges of the 21st century have been forecast and are well-known. In many ways we are already experiencing the future now. But predictions are hard to validate. A way around that is turning to slightly older predictions to illuminate the magnitude of the issues and the reality of their propositions.1

A new training model developed by MIT and Microsoft can help identify and correct an autonomous car's AI when it makes potentially deadly mistakes. Since their introduction several years ago, autonomous vehicles have slowly been making their way onto the road in greater and greater numbers, but the public remains wary of them despite the undeniable safety advantages they offer the public. Autonomous vehicle companies are fully aware of the public's skepticism. Every crash makes it more difficult to gain public trust and the fear is that if companies do not manage the autonomous vehicle roll-out properly, the backlash might close the door on self-driving car technology the way the Three Mile Island accident shut down the growth of nuclear power plants in the United States in the 1970's. Making autonomous vehicles safer than they already are means identifying those cases that programmers might never have thought of and that the AI will fail to respond to appropriately, but that a human driver will understand intuitively as a potentially dangerous situation.

A new telescope will take a sequence of hi-res snapshots with the world's largest digital camera, covering the entire visible night sky every few days - and repeating the process for an entire decade. That presents a big data challenge: What's the best way to rapidly and automatically identify and categorize all of the stars, galaxies, and other objects captured in these images? To help solve this problem, the scientific collaboration that is working on this Large Synoptic Survey Telescope project launched a competition among data scientists to train computers on how to best perform this task. The Photometric LSST Astronomical Time-Series Classification Challenge (PLAsTiCC), hosted on the Kaggle.com Kyle Boone, a UC Berkeley graduate student who has been working on computer algorithms in support of the Nearby Supernova Factory experiment and Supernova Cosmology Project efforts at the Department of Energy's Lawrence Berkeley National Laboratory (Berkeley Lab), devoted some of his spare time to the international machine-learning challenge in late 2018 while also working toward his Ph.D. "As I worked on job applications I started playing around with this competition to teach myself more about machine learning."

We propose a new framework for constructing polar codes (i.e., selecting the frozen bit positions) for arbitrary channels, and tailored to a given decoding algorithm, rather than based on the (not necessarily optimal) assumption of successive cancellation (SC) decoding. The proposed framework is based on the Genetic Algorithm (GenAlg), where populations (i.e., collections) of information sets evolve successively via evolutionary transformations based on their individual error-rate performance. These populations converge towards an information set that fits both the decoding behavior and the defined channel. Using our proposed algorithm over the additive white Gaussian noise (AWGN) channel, we construct a polar code of length 2048 with code rate 0.5, without the CRC-aid, tailored to plain successive cancellation list (SCL) decoding, achieving the same error-rate performance as the CRC-aided SCL decoding, and leading to a coding gain of 1 dB at BER of $10^{-6}$. Further, a belief propagation (BP)-tailored construction approaches the SCL error-rate performance without any modifications in the decoding algorithm itself. The performance gains can be attributed to the significant reduction in the total number of low-weight codewords. To demonstrate the flexibility, coding gains for the Rayleigh channel are shown under SCL and BP decoding. Besides improvements in error-rate performance, we show that, when required, the GenAlg can be also set up to reduce the decoding complexity, e.g., the SCL list size or the number of BP iterations can be reduced, while maintaining the same error-rate performance.

We propose a contextual-bandit approach for demand side management by offering price incentives. More precisely, a target mean consumption is set at each round and the mean consumption is modeled as a complex function of the distribution of prices sent and of some contextual variables such as the temperature, weather, and so on. The performance of our strategies is measured in quadratic losses through a regret criterion. We offer $\sqrt{T}$ upper bounds on this regret (up to poly-logarithmic terms), for strategies inspired by standard strategies for contextual bandits (like LinUCB, Li et al., 2010). Simulations on a real data set gathered by UK Power Networks, in which price incentives were offered, show that our strategies are effective and may indeed manage demand response by suitably picking the price levels.

Royal Dutch Shell is heavily investing in research and development of artificial intelligence (AI), which it hopes will provide solutions to some of its most pressing challenges. From meeting the demands of a transitioning energy market, urgently in need of cleaner and more efficient power, to improving safety on the forecourts of its service stations, AI is at the top of the agenda. I have been working with Shell over the past months to help create a data strategy, which gave me a thorough insight into Shell's AI priorities and initiatives. Current initiatives include deploying reinforcement learning in its exploration and drilling program, to reduce the cost of extracting the gas that still drives a significant proportion of its revenues. Elsewhere across its global business, Shell is rolling out AI at its public electric car charging stations, to manage the shifting demand for power throughout a day.

There is a really interesting concept in psychology called the Johari Window and it suggests that we rarely see ourselves as we actually are. Not only that, but we think other people see us differently than they do. Maybe you're not as charming as we think we are. Maybe that laughter after you told a joke was nervous laughter, but you thought you were hilarious. The key takeaway is that it's rare for people to accurately describe themselves to others as they actually are.

We consider the networked multi-agent reinforcement learning (MARL) problem in a fully decentralized setting, where agents learn to coordinate to achieve the joint success. This problem is widely encountered in many areas including traffic control, distributed control, and smart grids. We assume that the reward function for each agent can be different and observed only locally by the agent itself. Furthermore, each agent is located at a node of a communication network and can exchanges information only with its neighbors. Using softmax temporal consistency and a decentralized optimization method, we obtain a principled and data-efficient iterative algorithm. In the first step of each iteration, an agent computes its local policy and value gradients and then updates only policy parameters. In the second step, the agent propagates to its neighbors the messages based on its value function and then updates its own value function. Hence we name the algorithm value propagation. We prove a non-asymptotic convergence rate 1/T with the nonlinear function approximation. To the best of our knowledge, it is the first MARL algorithm with convergence guarantee in the control, off-policy and non-linear function approximation setting. We empirically demonstrate the effectiveness of our approach in experiments.

The goal of reinforcement learning is to construct algorithms that learn and plan in sequential decision making systems when the underlying system dynamics are unknown. A typical model in RL is Markov Decision Process (MDP). At each time step, the environment is in state s. The agent may take an action a, obtain a reward, and then the environment may transit to another state. In reinforcement learning, the transition probability distribution is unknown. The algorithm needs to learn the transition dynamics of MDP, while aiming to maximize the cumulative reward. This causes an exploration-exploitation dilemma: whether to act to gain new information (explore) or to act consistently with past experience to maximize reward (exploit). Theoretical analysis of reinforcement learning falls into two broad categories: those assuming a simulator (a.k.a.

Modern networks are causing a seismic shift in how real-time communications traverse IP networks to take the most optimal paths. Previous-generation techniques to manage traffic required static "if X, then Y" scenarios to be preprogrammed into networks on a hop-by-hop basis using legacy quality of service. But thanks to advancements in machine learning and AI, networks can take advantage of end-to-end network visibility and dynamic rerouting of data flows to dramatically improve real-time communications traffic performance and reliability. Legacy networks rely on traditional quality of service (QoS) to help improve the reliability of real-time communication data flows, such as voice and video. QoS uses a three-step process of identification, marking and policy enforcement to give preferential treatment to critical flows, including real-time streaming applications.