Despite the success of single-agent reinforcement learning, multi-agent reinforcement learning (MARL) remains challenging due to complex interactions between agents. Motivated by decentralized applications such as sensor networks, swarm robotics, and power grids, we study policy evaluation in MARL, where agents with jointly observed state-action pairs and private local rewards collaborate to learn the value of a given policy. In this paper, we propose a double averaging scheme, where each agent iteratively performs averaging over both space and time to incorporate neighboring gradient information and local reward information, respectively. We prove that the proposed algorithm converges to the optimal solution at a global geometric rate. In particular, such an algorithm is built upon a primal-dual reformulation of the mean squared projected Bellman error minimization problem, which gives rise to a decentralized convex-concave saddle-point problem. To the best of our knowledge, the proposed double averaging primal-dual optimization algorithm is the first to achieve fast finite-time convergence on decentralized convex-concave saddle-point problems.

Nokia last week said that it was selling off its digital health business, after failing to develop it into a substantial business itself, but this week the Finnish company is announcing an acquisition that underscores how it is doubling down in another one of its business areas, the Internet of Things. Nokia has acquired SpaceTime Insight, a California-based IoT startup that provides predictive analytics based on machine learning algorithms. Terms of the deal are not being disclosed, Bhaskar Gorti, president of Nokia Software, said in an interview. The startup had raised between $50 million and $65 million in funding (based on figures from Crunchbase and PitchBook), and PitchBook last estimated its valuation at just over $103 million in 2016. Backers of the startup included the energy giant E.ON, Novus Energy Partners, Zouk Capital and more.

Electricity is produced by a variety of generating units, each with different lead times and costs to be readied for service, and production costs once brought online. Because electricity is a commodity that cannot be easily stored, generation should match consumption at any given time; therefore, the cost of generating electricity has a direct relationship to electricity demand, typically referred to as electricity load. An accurate load forecast enables generators to optimize the mix of generating units that can serve the expected load while minimizing the production costs. This holds true for generators in both regulated and deregulated markets. In several deregulated markets, the electricity market operator is in charge of dispatching the available generation units according to the market's expected load and individual units' offered generation costs.

The energy industry has undergone many changes over the past couple of years. With new advances in industrial processes, multiple sources of energy are now available which can drive forward a more efficient use of energy resources. With the newest breakthroughs artificial intelligence (AI) has offered in the fields of robotics, self-driven cars, finance, and healthcare, energy companies are now exploring the possibilities of incorporating AI to increase the prospects of more efficient consumption of energy. Several artificial intelligence courses are already being developed to facilitate learning in the field of AI. The ability to compress and analyze large sets of data can help brands monitor and interpret the data produced by energy industries to optimize energy consumption.

In search applications, autonomous unmanned vehicles must be able to efficiently reacquire and localize mobile targets that can remain out of view for long periods of time in large spaces. As such, all available information sources must be actively leveraged -- including imprecise but readily available semantic observations provided by humans. To achieve this, this work develops and validates a novel collaborative human-machine sensing solution for dynamic target search. Our approach uses continuous partially observable Markov decision process (CPOMDP) planning to generate vehicle trajectories that optimally exploit imperfect detection data from onboard sensors, as well as semantic natural language observations that can be specifically requested from human sensors. The key innovation is a scalable hierarchical Gaussian mixture model formulation for efficiently solving CPOMDPs with semantic observations in continuous dynamic state spaces. The approach is demonstrated and validated with a real human-robot team engaged in dynamic indoor target search and capture scenarios on a custom testbed.

Their computer algorithm starts with abundant data about the proteins and metabolites in a biofuel-producing microbial pathway, but no information about how the pathway actually works. It then uses data from previous experiments to learn how the pathway will behave. The scientists used the technique to automatically predict the amount of biofuel produced by pathways that have been added to E. coli bacterial cells. The new approach is much faster than the current way to predict the behavior of pathways, and promises to speed up the development of biomolecules for many applications in addition to commercially viable biofuels, such as drugs that fight antibiotic-resistant infections and crops that withstand drought. The research is published May 29 in the journal npj Systems Biology and Applications. In biology, a pathway is a series of chemical reactions in a cell that produce a specific compound.

For CIOs and CTOs, asking which computing approaches add up to artificial intelligence and which are simply automation or BI is probably not a very useful question. The better question to think about: Do the latest developments in AI and machine learning provide a step change for solving problems and building new products or processes? David Gledhill, group CIO and head of technology and operations at DBS Bank in Singapore, put it this way: "There's a continuum. And moving along that continuum is what we care about," he said. "We'll leave it to the philosophers to determine what intelligence is."

Researchers from the U.S. Department of Energy's Lawrence Berkeley National Laboratory have created a new method using machine learning to accelerate the design of microbes that produce biofuel. To speed up the production of biofuels, the scientists developed a computer algorithm that begins with abundant data about the proteins and metabolites in a biofuel-producing microbial. However, the algorithm does not contain information about how the pathway actually work and instead uses data from previous experiments to learn how the pathway will behave. This new technique enables scientists to automatically predict the amount of biofuel produced by pathways that have been added to E. coli bacterial cells. A pathway is a series of chemical reactions in a cell that produce a specific compound, which researchers have sought to find ways to re-engineer and import from one microbe to another.

For instance, any of the utility company wants to have a close look of the remote power lines, the Airbus aerial satellites would help pull off data and for the closer look; it might contract with a local company to run a plane or drone flight over the area. Lately, Airbus announced its partnership with DroneBase for better results and inspection. Airbus has also started mapping the runways at Atlanta's Hartsfield-Jackson, the world's busiest airports. Here, the company used Sensefly's fixed-wing drone which flies autonomously, capturing images of the ground, and then the results are checked and complied, including 3D maps to show bumps and cracks, and GPS data to locate busted lights. "In the coming time, drones would be taking up certain projects that would make a great impact," says FAA.

IMAGE: A new approach developed by Zak Costello (left) and Hector Garcia Martin brings the the speed and analytic power of machine learning to bioengineering. Scientists from the Department of Energy's Lawrence Berkeley National Laboratory (Berkeley Lab) have developed a way to use machine learning to dramatically accelerate the design of microbes that produce biofuel. Their computer algorithm starts with abundant data about the proteins and metabolites in a biofuel-producing microbial pathway, but no information about how the pathway actually works. It then uses data from previous experiments to learn how the pathway will behave. The scientists used the technique to automatically predict the amount of biofuel produced by pathways that have been added to E. coli bacterial cells.