Energy
Resource-Constrained On-Device Learning by Dynamic Averaging
Heppe, Lukas, Kamp, Michael, Adilova, Linara, Heinrich, Danny, Piatkowski, Nico, Morik, Katharina
The communication between data-generating devices is partially responsible for a growing portion of the world's power consumption. Thus reducing communication is vital, both, from an economical and an ecological perspective. For machine learning, on-device learning avoids sending raw data, which can reduce communication substantially. Furthermore, not centralizing the data protects privacy-sensitive data. However, most learning algorithms require hardware with high computation power and thus high energy consumption. In contrast, ultra-low-power processors, like FPGAs or micro-controllers, allow for energy-efficient learning of local models. Combined with communication-efficient distributed learning strategies, this reduces the overall energy consumption and enables applications that were yet impossible due to limited energy on local devices. The major challenge is then, that the low-power processors typically only have integer processing capabilities. This paper investigates an approach to communication-efficient on-device learning of integer exponential families that can be executed on low-power processors, is privacy-preserving, and effectively minimizes communication. The empirical evaluation shows that the approach can reach a model quality comparable to a centrally learned regular model with an order of magnitude less communication. Comparing the overall energy consumption, this reduces the required energy for solving the machine learning task by a significant amount.
Five Key Factors for a Future-Oriented Digital Transformation of Electric Power Enterprises
At HUAWEI CONNECT 2020, IDC and Huawei jointly released the white paper for the electric power industry -- Building the Future-Ready Power Enterprise: Road to a Successful Digital Transformation. In the white paper, IDC proposed a methodology for the transformation of electric power enterprises. This methodology supports and aligns with Huawei's digital transformation methodology. IDC and Huawei follow a similar approach with frameworks and blueprints to help organizations design their digital transformation priorities and set their agenda, which in turn enables power enterprises to deliver business value of scale. The power industry has long faced disruption. Power enterprises are now facing multiple changes.
Microsoft And Shell Announce New Partnership To Use Artificial Intelligence And Tech To Reduce Carbon Emissions
Tackling carbon emissions is one of the biggest challenges faced by the world today. For big business, this means making a strategic and managed move towards increasing the use of renewable energy sources, as well as creating efficiencies across all aspects of their operations. It's a difficult task to manage alone, even for an enterprise on the scale of tech giant Microsoft or energy titan Shell. But working together creates new possibilities that go further than what it is likely they could accomplish individually. Beyond meeting their own zero-carbon commitments, there's the opportunity to help other companies within their vast ecosystems of customers and suppliers to meet their environmental and safety goals, too.
Sizing up a green carbon sink
Forests are having their moment. Because trees can vacuum carbon from the atmosphere and lock it away in wood, governments and businesses are embracing efforts to fight climate change by reforesting cleared areas and planting trees on a massive scale. But scientists have warned that the enthusiasm and money flowing to forest-based climate solutions threaten to outpace the science. Two papers published this week seek to put such efforts on a firmer footing. One study quantifies how much carbon might be absorbed globally by allowing forests cleared for farming or other purposes to regrow. The other calculates how much carbon could be sequestered by forests in the United States if they were fully โstockedโ with newly planted trees. Each strategy has promise, the studies suggest, but also faces perils. To get a worldwide perspective on the potential of second-growth forests, an international team led by ecologist Susan Cook-Patton of the Nature Conservancy (TNC) assembled data from more than 13,000 previously deforested sites where researchers had measured regrowth rates of young trees. The team then trained a machine-learning algorithm on those data and dozens of variables, such as climate and soil type, to predict and map how fast trees could grow on other cleared sites where it didn't have data. > Can the forest regenerate naturally, or can we do something to help? > > Susan Cook-Patton , the Nature Conservancy A TNC-led team had previously calculated that some 678 million hectares, an area nearly the size of Australia, could support second-growth forests. (The total doesn't include land where trees might not be desirable, such as farmland and ecologically valuable grasslands.) If trees were allowed to take over that entire area, new forests could soak up one-quarter of the world's fossil fuel emissions over the next 30 years, Cook-Patton and colleagues report in Nature . That absorption rate is 32% higher than a previous estimate, based on coarser data, produced by the Intergovernmental Panel on Climate Change. But the total carbon drawdown is 11% lower than a TNC-led team estimated in 2017. The study highlights โwhat nature can do all on its own,โ Cook-Patton says. And it represents โa lightning step forwardโ in precision compared with earlier studies, says geographer Matthew Fagan of the University of Maryland, Baltimore County, who was not involved in the work. But, Fagan adds, โNatural regrowth is not going to save the planet.โ One problem: There is often little economic incentive for private landowners to allow forests to bounce back. Under current policies and market pricing, โnobody will abandon cattle ranching or agriculture for growing carbon,โ says Pedro Brancalion, a forest expert at the University of Sรฃo Paulo in Piracicaba, Brazil. And even when forests get a second life, they often don't last long enough to store much carbon before being cleared again. Fagan notes that even in Costa Rica, renowned as a reforestation champion for doubling its forest cover in recent decades, studies have found that half of second-growth forests fall within 20 years. Given such realities, some advocates are pushing to expand tree planting in existing forests. To boost that concept, a team of researchers at the U.S. Forest Service (USFS) quantified how many additional trees U.S. forests could hold. Drawing on a federal inventory, they found that more than 16% of forests in the continental United States are โunderstockedโโholding fewer than 35% of the trees they could support. Fully stocking these 33 million hectares of forest would ultimately enable U.S. forests to sequester about 18% of national carbon emissions each year, up from 15% today, the team reports in the Proceedings of the National Academy of Sciences . But for that to happen, the United States would have to โmassivelyโ expand its annual tree-planting efforts, from about 1 billion to 16 billion trees, says lead author Grant Domke, a USFS research forester in St. Paul, Minnesota. Cook-Patton says planting trees might make sense in some places, but natural regeneration, where possible, provides more bang for the buck. โFor any given site,โ she says, โwe should always ask ourselves first: โCan the forest regenerate naturally, or can we do something to help?โโ
The Future of Atoms: Artificial Intelligence for Nuclear Applications
Held virtually today on the sidelines of the 64th IAEA General Conference, the first ever IAEA meeting discussing the use of artificial intelligence (AI) for nuclear applications showcased the ways in which AI-based approaches in nuclear science can benefit human health, water resource management and nuclear fusion research. Open to the public, the event gathered over 300 people from 43 countries and launched a global dialogue on the potential of AI for nuclear science and the related implications of its use, including ethics and transparency. AI refers to a collection of technologies that combine numerical data, process algorithms and continuously increasing computing power to develop systems capable of tracking complex problems in ways similar to human logic and reasoning. AI technologies can analyse large amounts of data to "learn" how to complete a particular task, a technique called machine learning. "Artificial Intelligence is advancing exponentially," said Najat Mokhtar, IAEA Deputy Director General and Head of the Department of Nuclear Sciences and Applications.
The History, Status, and Future of FPGAs
This article is a summary of a three-hour discussion at Stanford University in September 2019 among the authors. It has been written with combined experiences at and with organizations such as Zilog, Altera, Xilinx, Achronix, Intel, IBM, Stanford, MIT, Berkeley, University of Wisconsin, the Technion, Fairchild, Bell Labs, Bigstream, Google, DIGITAL (DEC), SUN, Nokia, SRI, Hitachi, Silicom, Maxeler Technologies, VMware, Xerox PARC, Cisco, and many others. These organizations are not responsible for the content, but may have inspired the authors in some ways, to arrive at the colorful ride through FPGA space described here. Field-programmable gate arrays (FPGAs) have been hitting a nerve in the ASIC community since their inception. In the mid-1980s, Ross Freeman and his colleagues bought the technology from Zilog and started Xilinx, targeting the ASIC emulation and education markets.
Continual Model-Based Reinforcement Learning with Hypernetworks
Huang, Yizhou, Xie, Kevin, Bharadhwaj, Homanga, Shkurti, Florian
Lifelong model-based robot learning is predicated upon continual adaptation to the dynamics of new tasks. For example, robots need to learn to manipulate unseen objects with various mass distributions, walk on new types of terrains with different friction, elasticity, and other physical properties, or even learn to adapt to different tasks, such as walking, running, or climbing stairs. This presents at least two challenges for many model-based reinforcement learning (MBRL) and model-predictive control (MPC) formulations, which typically comprise of a dynamics learning phase followed by a planning/policy optimization and execution phase. First, these methods are not scalable because the time required to train the dynamics model grows linearly with the size of the collected experience. Second, as the robot learner encounters and adapts to new tasks, it has to avoid catastrophic forgetting of the dynamics of old tasks, and should ideally exhibit both forward transfer (old tasks improve the learning performance on the new task) and backward transfer (new task improves the performance on old tasks). Many MBRL and MPC methods lack this type of adaptation and positive transfer. In this work, we propose to extend the task-aware continual learning approach based on hypernetworks in [1] to adapt to changing environment dynamics and to address the scalability and positive transfer challenges mentioned above in a reinforcement learning setting.
A physics-informed operator regression framework for extracting data-driven continuum models
Patel, Ravi G., Trask, Nathaniel A., Wood, Mitchell A., Cyr, Eric C.
The application of deep learning toward discovery of data-driven models requires careful application of inductive biases to obtain a description of physics which is both accurate and robust. We present here a framework for discovering continuum models from high fidelity molecular simulation data. Our approach applies a neural network parameterization of governing physics in modal space, allowing a characterization of differential operators while providing structure which may be used to impose biases related to symmetry, isotropy, and conservation form. We demonstrate the effectiveness of our framework for a variety of physics, including local and nonlocal diffusion processes and single and multiphase flows. For the flow physics we demonstrate this approach leads to a learned operator that generalizes to system characteristics not included in the training sets, such as variable particle sizes, densities, and concentration.
Adversarial Examples in Deep Learning for Multivariate Time Series Regression
Mode, Gautam Raj, Hoque, Khaza Anuarul
Multivariate time series (MTS) regression tasks are common in many real-world data mining applications including finance, cybersecurity, energy, healthcare, prognostics, and many others. Due to the tremendous success of deep learning (DL) algorithms in various domains including image recognition and computer vision, researchers started adopting these techniques for solving MTS data mining problems, many of which are targeted for safety-critical and cost-critical applications. Unfortunately, DL algorithms are known for their susceptibility to adversarial examples which also makes the DL regression models for MTS forecasting also vulnerable to those attacks. To the best of our knowledge, no previous work has explored the vulnerability of DL MTS regression models to adversarial time series examples, which is an important step, specifically when the forecasting from such models is used in safety-critical and cost-critical applications. In this work, we leverage existing adversarial attack generation techniques from the image classification domain and craft adversarial multivariate time series examples for three state-of-the-art deep learning regression models, specifically Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU). We evaluate our study using Google stock and household power consumption dataset. The obtained results show that all the evaluated DL regression models are vulnerable to adversarial attacks, transferable, and thus can lead to catastrophic consequences in safety-critical and cost-critical domains, such as energy and finance.