Most deep learning models are based on deep neural networks with multiple layers between input and output. The parameters defining these layers are initialized using random values and are "learned" from data, typically using stochastic gradient descent based algorithms. These algorithms rely on data being randomly shuffled before optimization. The randomization of the data prior to processing in batches that is formally required for stochastic gradient descent algorithm to effectively derive a useful deep learning model is expected to be prohibitively expensive for in situ model training because of the resulting data communications across the processor nodes. We show that the stochastic gradient descent (SGD) algorithm can still make useful progress if the batches are defined on a per-processor basis and processed in random order even though (i) the batches are constructed from data samples from a single class or specific flow region, and (ii) the overall data samples are heterogeneous. We present block-random gradient descent, a new algorithm that works on distributed, heterogeneous data without having to pre-shuffle. This algorithm enables in situ learning for exascale simulations. The performance of this algorithm is demonstrated on a set of benchmark classification models and the construction of a subgrid scale large eddy simulations (LES) model for turbulent channel flow using a data model similar to that which will be encountered in exascale simulation.

Google's DeepMind is using machine learning to predict the performance of its wind turbines 36 hours in advance. The prediction of wind turbine performance for turbines in the central United States more than a day in advance has led to a roughly 20 percent increase in the value of wind energy, Google and DeepMind said in a joint blog post today. The model is trained using weather data and historical wind turbine performance data. "Based on these predictions, our model recommends how to make optimal hourly delivery commitments to the power grid a full day in advance. This is important, because energy sources that can be scheduled (i.e. can deliver a set amount of electricity at a set time) are often more valuable to the grid," the post reads.

The Azeri-Chirag-Deepwater Gunashli (ACG), a sprawling complex of offshore oil fields 60 miles off Azerbaijan's capital Baku, is causing somewhat of a headache for BP's head of technology. "We have huge production in Azerbaijan of wells that are quite prone to producing sand, and sand if it's produced in high quantities from our oil wells can do damage to the metalwork and also choke back the production," says David Eyton. The ACG, which pumps out an average of 584,000 barrels of oil per day, is a prized asset for BP, and any hold ups could cost the company dearly. But the man leading BP's technology revolution think he has a solution: artificial intelligence (AI).

Active learning methods for neural networks are usually based on greedy criteria which ultimately give a single new design point for the evaluation. Such an approach requires either some heuristics to sample a batch of design points at one active learning iteration, or retraining the neural network after adding each data point, which is computationally inefficient. Moreover, uncertainty estimates for neural networks sometimes are overconfident for the points lying far from the training sample. In this work we propose to approximate Bayesian neural networks (BNN) by Gaussian processes, which allows us to update the uncertainty estimates of predictions efficiently without retraining the neural network, while avoiding overconfident uncertainty prediction for out-of-sample points. In a series of experiments on real-world data including large-scale problems of chemical and physical modeling, we show superiority of the proposed approach over the state-of-the-art methods.

We consider the band assignment (BA) problem in dual-band systems, where the basestation (BS) chooses one of the two available frequency bands (centimeter-wave and millimeter-wave bands) to communicate with the user equipment (UE). While the millimeter-wave band might offer higher data rate, there is a significant probability of outage during which the communication should be carried on the (more reliable) centimeter-wave band. We consider two variations of the BA problem, one-shot and sequential BA. For the former the BS uses only the currently observed information to decide whether to switch to the other frequency band, for the sequential BA, the BS uses a window of previously observed information to predict the best band for a future time step. We provide two approaches to solve the BA problem, (i) a deep learning approach that is based on Long Short Term Memory and/or multi-layer Neural Networks, and (ii) a Gaussian Process based approach, which relies on the assumption that the channel states are jointly Gaussian. We compare the achieved performances to several benchmarks in two environments: (i) a stochastic environment, and (ii) microcellular outdoor channels obtained by ray-tracing. In general, the deep learning solution shows superior performance in both environments.

One endeavour of modern physical chemistry is to use bottom-up approaches to design materials and drugs with desired properties. Here we introduce an atomistic structure learning algorithm (ASLA) that utilizes a convolutional neural network to build 2D compounds and layered structures atom by atom. The algorithm takes no prior data or knowledge on atomic interactions but inquires a first-principles quantum mechanical program for physical properties. Using reinforcement learning, the algorithm accumulates knowledge of chemical compound space for a given number and type of atoms and stores this in the neural network, ultimately learning the blueprint for the optimal structural arrangement of the atoms for a given target property. ASLA is demonstrated to work on diverse problems, including grain boundaries in graphene sheets, organic compound formation and a surface oxide structure. This approach to structure prediction is a first step toward direct manipulation of atoms with artificially intelligent first principles computer codes.

Carbon-free technologies like renewable energy help combat climate change, but many of them have not reached their full potential. Consider wind power: over the past decade, wind farms have become an important source of carbon-free electricity as the cost of turbines has plummeted and adoption has surged. However, the variable nature of wind itself makes it an unpredictable energy source--less useful than one that can reliably deliver power at a set time. In search of a solution to this problem, last year DeepMind and Google started applying machine learning algorithms to 700 megawatts of wind power capacity in the central United States. These wind farms--part of Google's global fleet of renewable energy projects--collectively generate as much electricity as is needed by a medium-sized city.

Wind power has become increasingly popular, but its success is limited by the fact that wind comes and goes as it pleases, making it hard for power grids to count on the renewable energy and less likely to fully embrace it. While we can't control the wind, Google has an idea for the next best thing: using machine learning to predict it. Google and DeepMind have started testing machine learning on Google's own wind turbines, which are part of the company's renewable energy projects. Beginning last year, they fed weather forecasts and existing turbine data into DeepMind's machine learning platform, which churned out wind power predictions 36 hours ahead of actual power generation. Google could then make supply commitments to power grids a full day before delivery.

Robots have walked on legs for decades. Today's most advanced humanoid robots can tramp along flat and inclined surfaces, climb up and down stairs, and slog through rough terrain. But despite the progress, legged robots still can't begin to match the agility, efficiency, and robustness of humans and animals. Existing walking robots hog power and spend too much time in the shop. All too often, they fail, they fall, and they break. For the robotic helpers we've long dreamed of to become a reality, these machines will have to learn to walk as we do. We must build robots with legs because our world is designed for legs.

When I was a kid, I used to watch cartoons as I got dressed for school. My favorite was The Jetsons: the flying cars, the robot maid, food served hot at the touch of a button. The only thing I could never figure out was why – despite all the seemingly futuristic advances – George still went to work every day and pushed a bunch of buttons? It stayed with me, even as I got older, and when I'd find an old episode of The Jetsons on cable, it would really vex me. If a robot could do the household chores and give advice to Judy and little Elroy – couldn't a robot make Spacely Sprockets?