Energy
Bayesian Numerical Integration with Neural Networks
Ott, Katharina, Tiemann, Michael, Hennig, Philipp, Briol, François-Xavier
Bayesian probabilistic numerical methods for numerical integration offer significant advantages over their non-Bayesian counterparts: they can encode prior information about the integrand, and can quantify uncertainty over estimates of an integral. However, the most popular algorithm in this class, Bayesian quadrature, is based on Gaussian process models and is therefore associated with a high computational cost. To improve scalability, we propose an alternative approach based on Bayesian neural networks which we call Bayesian Stein networks. The key ingredients are a neural network architecture based on Stein operators, and an approximation of the Bayesian posterior based on the Laplace approximation. We show that this leads to orders of magnitude speed-ups on the popular Genz functions benchmark, and on challenging problems arising in the Bayesian analysis of dynamical systems, and the prediction of energy production for a large-scale wind farm.
Approximating ReLU on a Reduced Ring for Efficient MPC-based Private Inference
Secure multi-party computation (MPC) allows users to offload machine learning inference on untrusted servers without having to share their privacy-sensitive data. Despite their strong security properties, MPC-based private inference has not been widely adopted in the real world due to their high communication overhead. When evaluating ReLU layers, MPC protocols incur a significant amount of communication between the parties, making the end-to-end execution time multiple orders slower than its non-private counterpart. This paper presents HummingBird, an MPC framework that reduces the ReLU communication overhead significantly by using only a subset of the bits to evaluate ReLU on a smaller ring. Based on theoretical analyses, HummingBird identifies bits in the secret share that are not crucial for accuracy and excludes them during ReLU evaluation to reduce communication. With its efficient search engine, HummingBird discards 87--91% of the bits during ReLU and still maintains high accuracy. On a real MPC setup involving multiple servers, HummingBird achieves on average 2.03--2.67x end-to-end speedup without introducing any errors, and up to 8.64x average speedup when some amount of accuracy degradation can be tolerated, due to its up to 8.76x communication reduction.
Non-linear dimension reduction in factor-augmented vector autoregressions
The COVID-19 pandemic belongs to the severest health, economic and social crises in recent decades and poses the greatest challenge to the world economy since World War II. The virus has spread around the globe and paralyzed entire economic sectors and activities. For economic modeling, the COVID-19 pandemic entails dealing with huge, unprecedented outliers in datasets which adversely affect the reliability of established, mostly linear, economic models. To the detriment of those commonly used models, economic indicators and variables are prone to unanticipated movements and do not respond in the way they are supposed to. Large shifts in the level of certain variables and strong deviations from their usual paths clearly aggravate the challenge of handling large outliers within existing econometric models.
A Spatiotemporal Deep Neural Network for Fine-Grained Multi-Horizon Wind Prediction
Huang, Fanling, Deng, Yangdong
The prediction of wind in terms of both wind speed and direction, which has a crucial impact on many real-world applications like aviation and wind power generation, is extremely challenging due to the high stochasticity and complicated correlation in the weather data. Existing methods typically focus on a sub-set of influential factors and thus lack a systematic treatment of the problem. In addition, fine-grained forecasting is essential for efficient industry operations, but has been less attended in the literature. In this work, we propose a novel data-driven model, Multi-Horizon SpatioTemporal Network (MHSTN), generally for accurate and efficient fine-grained wind prediction. MHSTN integrates multiple deep neural networks targeting different factors in a sequence-to-sequence (Seq2Seq) backbone to effectively extract features from various data sources and produce multi-horizon predictions for all sites within a given region. MHSTN is composed of four major modules. First, a temporal module fuses coarse-grained forecasts derived by Numerical Weather Prediction (NWP) and historical on-site observation data at stations so as to leverage both global and local atmospheric information. Second, a spatial module exploits spatial correlation by modeling the joint representation of all stations. Third, an ensemble module weighs the above two modules for final predictions. Furthermore, a covariate selection module automatically choose influential meteorological variables as initial input. MHSTN is already integrated into the scheduling platform of one of the busiest international airports of China. The evaluation results demonstrate that our model outperforms competitors by a significant margin.
Three top takeaways from the Senate Energy committee hearing on DOE and AI
Fox News correspondent Gillian Turner has the latest on the president's focus amid calls for an impeachment inquiry on'Special Report.' Lawmakers on the Senate Energy Committee were warned on Thursday about both the threats and opportunities that come with artificial intelligence being integrated into the U.S. energy sector and everyday life as a whole. The committee held a hearing on the rapidly advancing technology, and experts present spent a significant amount of time not only discussing AI but the ever-looming threat of China and its efforts to steal and recreate emerging U.S. capabilities. "China released their new generation of AI Development Plan, which includes [research and development] and infrastructure targets. The U.S. currently does not have a strategic AI plan like this," Committee Chair Joe Manchin, D-W.Va., said at the hearing's outset.
Instance Segmentation in the Dark
Chen, Linwei, Fu, Ying, Wei, Kaixuan, Zheng, Dezhi, Heide, Felix
Noname manuscript No. (will be inserted by the editor) Abstract Existing instance segmentation techniques are primarily depth can be critical for low-light instance segmentation. To tailored for high-visibility inputs, but their performance mitigate the scarcity of annotated RAW datasets, we leverage significantly deteriorates in extremely low-light environments. In addition, to facilitate further research in the dark and introduce several techniques that in this direction, we capture a real-world low-light instance substantially boost the low-light inference accuracy. The proposed segmentation dataset comprising over two thousand paired method is motivated by the observation that noise in low/normal-light images with instance-level pixel-wise annotations. To suppress this "feature noise", we in very low light (4 % AP higher than state-of-the-art propose a novel learning method that relies on an adaptive competitors), meanwhile opening new opportunities for future weighted downsampling layer, a smooth-oriented convolutional research. Our code and dataset are publicly available to block, and disturbance suppression learning. Furthermore, we discover that high-bit-depth RAW images can better preserve richer scene information in low-light conditions compared 1 Introduction to typical camera sRGB outputs, thus supporting the use of RAW-input algorithms. "buried" by severe noise caused by limited photon count and They substantially improve the capability of models to learn noiseresisted features and thus boost the low-light segmentation accuracy appreciably. It is worth noting that they are modelagnostic and lightweight or even cost-free. It aggregates local features adaptively and suppresses the high-frequency disturbance caused by noise as well as keeping the details in deep features. The smoothoriented convolutional block enhances the ordinary convolutional layers by adding a smooth-oriented convolution branch. Relevant Moreover, we notice that the high bit-depth can be crucial low-light recognition/detection methods (Cui et al., 2021; for low-light conditions.
Quantum-Inspired Machine Learning: a Survey
Huynh, Larry, Hong, Jin, Mian, Ajmal, Suzuki, Hajime, Wu, Yanqiu, Camtepe, Seyit
Quantum-inspired Machine Learning (QiML) is a burgeoning field, receiving global attention from researchers for its potential to leverage principles of quantum mechanics within classical computational frameworks. However, current review literature often presents a superficial exploration of QiML, focusing instead on the broader Quantum Machine Learning (QML) field. In response to this gap, this survey provides an integrated and comprehensive examination of QiML, exploring QiML's diverse research domains including tensor network simulations, dequantized algorithms, and others, showcasing recent advancements, practical applications, and illuminating potential future research avenues. Further, a concrete definition of QiML is established by analyzing various prior interpretations of the term and their inherent ambiguities. As QiML continues to evolve, we anticipate a wealth of future developments drawing from quantum mechanics, quantum computing, and classical machine learning, enriching the field further. This survey serves as a guide for researchers and practitioners alike, providing a holistic understanding of QiML's current landscape and future directions.
Advanced Computing and Related Applications Leveraging Brain-inspired Spiking Neural Networks
Sima, Lyuyang, Bucukovski, Joseph, Carlson, Erwan, Yien, Nicole L.
In the rapid evolution of next-generation brain-inspired artificial intelligence and increasingly sophisticated electromagnetic environment, the most bionic characteristics and anti-interference performance of spiking neural networks show great potential in terms of computational speed, real-time information processing, and spatio-temporal information processing. Data processing. Spiking neural network is one of the cores of brain-like artificial intelligence, which realizes brain-like computing by simulating the structure and information transfer mode of biological neural networks. This paper summarizes the strengths, weaknesses and applicability of five neuronal models and analyzes the characteristics of five network topologies; then reviews the spiking neural network algorithms and summarizes the unsupervised learning algorithms based on synaptic plasticity rules and four types of supervised learning algorithms from the perspectives of unsupervised learning and supervised learning; finally focuses on the review of brain-like neuromorphic chips under research at home and abroad. This paper is intended to provide learning concepts and research orientations for the peers who are new to the research field of spiking neural networks through systematic summaries.
Multi-contact Stochastic Predictive Control for Legged Robots with Contact Locations Uncertainty
Gazar, Ahmad, Khadiv, Majid, Del Prete, Andrea, Righetti, Ludovic
Trajectory optimization under uncertainties is a challenging problem for robots in contact with the environment. Such uncertainties are inevitable due to estimation errors, control imperfections, and model mismatches between planning models used for control and the real robot dynamics. This induces control policies that could violate the contact location constraints by making contact at unintended locations, and as a consequence leading to unsafe motion plans. This work addresses the problem of robust kino-dynamic whole-body trajectory optimization using stochastic nonlinear model predictive control (SNMPC) by considering additive uncertainties on the model dynamics subject to contact location chance-constraints as a function of robot's full kinematics. We demonstrate the benefit of using SNMPC over classic nonlinear MPC (NMPC) for whole-body trajectory optimization in terms of contact location constraint satisfaction (safety). We run extensive Monte-Carlo simulations for a quadruped robot performing agile trotting and bounding motions over small stepping stones, where contact location satisfaction becomes critical. Our results show that SNMPC is able to perform all motions safely with 100% success rate, while NMPC failed 48.3% of all motions.
Leveraging the Potential of Novel Data in Power Line Communication of Electricity Grids
Balada, Christoph, Bondorf, Max, Ahmed, Sheraz, Dengela, Andreas, Zdrallek, Markus
Electricity grids have become an essential part of daily life, even if they are often not noticed in everyday life. We usually only become particularly aware of this dependence by the time the electricity grid is no longer available. However, significant changes, such as the transition to renewable energy (photovoltaic, wind turbines, etc.) and an increasing number of energy consumers with complex load profiles (electric vehicles, home battery systems, etc.), pose new challenges for the electricity grid. To address these challenges, we propose two first-of-its-kind datasets based on measurements in a broadband powerline communications (PLC) infrastructure. Both datasets FiN-1 and FiN-2, were collected during real practical use in a part of the German low-voltage grid that supplies around 4.4 million people and show more than 13 billion datapoints collected by more than 5100 sensors. In addition, we present different use cases in asset management, grid state visualization, forecasting, predictive maintenance, and novelty detection to highlight the benefits of these types of data. For these applications, we particularly highlight the use of novel machine learning architectures to extract rich information from real-world data that cannot be captured using traditional approaches. By publishing the first large-scale real-world dataset, we aim to shed light on the previously largely unrecognized potential of PLC data and emphasize machine-learning-based research in low-voltage distribution networks by presenting a variety of different use cases.