Energy
Learning Manifold Implicitly via Explicit Heat-Kernel Learning
Zhou, Yufan, Chen, Changyou, Xu, Jinhui
Manifold learning is a fundamental problem in machine learning with numerous applications. Most of the existing methods directly learn the low-dimensional embedding of the data in some high-dimensional space, and usually lack the flexibility of being directly applicable to down-stream applications. In this paper, we propose the concept of implicit manifold learning, where manifold information is implicitly obtained by learning the associated heat kernel. A heat kernel is the solution of the corresponding heat equation, which describes how "heat" transfers on the manifold, thus containing ample geometric information of the manifold. We provide both practical algorithm and theoretical analysis of our framework. The learned heat kernel can be applied to various kernel-based machine learning models, including deep generative models (DGM) for data generation and Stein Variational Gradient Descent for Bayesian inference. Extensive experiments show that our framework can achieve state-of-the-art results compared to existing methods for the two tasks.
"Drunk Man" Saves Our Lives: Route Planning by a Biased Random Walk Mode
Hu, Xinyi, Miao, Quchen, Zhao, Zexuan
Based on the hurricane struking Puerto Rico in 2017, we developed a transportable disaster response system "DroneGo" featuring a drone fleet capable of delivering medical package and videoing roads. Assuming equal weight for both mission, we take the capability of carrying out the former missions as a constraint and a starting point from which reconnaissance routes are built. The feasibility of fitting packages into cargo bay 1 or 2 is tested by genetic algorithm. In scenario where drones carry packages to and unloaded back, from specification of drones and loading weight can we derive the maximum reachable distance of each drone loaded. A k-means clustering algorithm is used for partitioning destinations and deriving centroids as locations of bases.
RAISE 2020: PM Modi to inaugurate global summit on artificial intelligence on October 5
New Delhi, Oct 02: Prime Minister Narendra Modi is all set to inaugurate a global virtual summit on Artificial Intelligence (AI), RAISE 2020 - 'Responsible AI for Social Empowerment 2020' on October 5, according to the ministry of electronics and information technology. According to reports, the summit is scheduled to be held from October 5 to 9, 2020 and is being organised by the ministry of electronics and information technology (MeitY) and NITI Aayog. Hathras case: Demand to impose President's rule in UP The inauguration event will take place in the presence of minister of electronics & IT, communications and law & justice, Ravi Shankar Prasad, eminent global AI expert Professor Raj Reddy, Reliance Industries Ltd chairman Mukesh Ambani, IBM CEO Arvind Krishna among others. Professor Raj Reddy will hold a session about developing voice-enabled AI that removes linguistic barriers on October 6, the second day of the summit. Former Infosys CFO Mohandas Pai, and Brad Smith, president & legal head, Microsoft Global will also participate in sessions.
AI-based traffic management improves mobility, saves fuel, cuts pollution -- GCN
According to the Department of Energy's 2020 Transportation Energy Data Book, the transportation sector is responsible for more than 69% of petroleum consumption. The Environmental Protection Agency says emissions from transportation account for about 28% of total U.S. greenhouse gas emissions. Not all that fuel is efficiently used, contributing to CO2 emissions without providing real benefits. Vehicles stopping for red lights, idling as they wait for the signal lights to change and accelerating to get back up to speed wastes fuel and adds pollutants to the air. Idling vehicles waste more than 6 billion gallons of gasoline and diesel combined every year, DOE estimates.
Disentangling causal effects for hierarchical reinforcement learning
Exploration and credit assignment under sparse rewards are still challenging problems. We argue that these challenges arise in part due to the intrinsic rigidity of operating at the level of actions. Actions can precisely define how to perform an activity but are ill-suited to describe what activity to perform. Instead, causal effects are inherently composable and temporally abstract, making them ideal for descriptive tasks. By leveraging a hierarchy of causal effects, this study aims to expedite the learning of task-specific behavior and aid exploration. Borrowing counterfactual and normality measures from causal literature, we disentangle controllable effects from effects caused by other dynamics of the environment. We propose CEHRL, a hierarchical method that models the distribution of controllable effects using a Variational Autoencoder. This distribution is used by a high-level policy to 1) explore the environment via random effect exploration so that novel effects are continuously discovered and learned, and to 2) learn task-specific behavior by prioritizing the effects that maximize a given reward function. In comparison to exploring with random actions, experimental results show that random effect exploration is a more efficient mechanism and that by assigning credit to few effects rather than many actions, CEHRL learns tasks more rapidly.
Adversarial and Natural Perturbations for General Robustness
Gulshad, Sadaf, Metzen, Jan Hendrik, Smeulders, Arnold
In this paper we aim to explore the general robustness of neural network classifiers by utilizing adversarial as well as natural perturbations. Different from previous works which mainly focus on studying the robustness of neural networks against adversarial perturbations, we also evaluate their robustness on natural perturbations before and after robustification. After standardizing the comparison between adversarial and natural perturbations, we demonstrate that although adversarial training improves the performance of the networks against adversarial perturbations, it leads to drop in the performance for naturally perturbed samples besides clean samples. In contrast, natural perturbations like elastic deformations, occlusions and wave does not only improve the performance against natural perturbations, but also lead to improvement in the performance for the adversarial perturbations. Additionally they do not drop the accuracy on the clean images. A large body of work in computer vision and machine learning research focuses on studying the robustness of neural networks against adversarial perturbations (Kurakin et al., 2016; Goodfellow et al., 2014; Carlini & Wagner, 2017). Various defense based methods have also been proposed against these adversarial perturbations (Goodfellow et al., 2014; Madry et al., 2017; Zhang et al., 2019b; Song et al., 2019).
Random Coordinate Langevin Monte Carlo
Ding, Zhiyan, Li, Qin, Lu, Jianfeng, Wright, Stephen J.
Langevin Monte Carlo (LMC) is a popular Markov chain Monte Carlo sampling method. One drawback is that it requires the computation of the full gradient at each iteration, an expensive operation if the dimension of the problem is high. We propose a new sampling method: Random Coordinate LMC (RC-LMC). At each iteration, a single coordinate is randomly selected to be updated by a multiple of the partial derivative along this direction plus noise, and all other coordinates remain untouched. We investigate the total complexity of RC-LMC and compare it with the classical LMC for log-concave probability distributions. When the gradient of the log-density is Lipschitz, RC-LMC is less expensive than the classical LMC if the log-density is highly skewed for high dimensional problems, and when both the gradient and the Hessian of the log-density are Lipschitz, RC-LMC is always cheaper than the classical LMC, by a factor proportional to the square root of the problem dimension. In the latter case, our estimate of complexity is sharp with respect to the dimension.
The energy distance for ensemble and scenario reduction
Scenario reduction techniques are widely applied for solving sophisticated dynamic and stochastic programs, especially in energy and power systems, but also used in probabilistic forecasting, clustering and estimating generative adversarial networks (GANs). We propose a new method for ensemble and scenario reduction based on the energy distance which is a special case of the maximum mean discrepancy (MMD). We discuss the choice of energy distance in detail, especially in comparison to the popular Wasserstein distance which is dominating the scenario reduction literature. The energy distance is a metric between probability measures that allows for powerful tests for equality of arbitrary multivariate distributions or independence. Thanks to the latter, it is a suitable candidate for ensemble and scenario reduction problems. The theoretical properties and considered examples indicate clearly that the reduced scenario sets tend to exhibit better statistical properties for the energy distance than a corresponding reduction with respect to the Wasserstein distance. We show applications to a Bernoulli random walk and two real data based examples for electricity demand profiles and day-ahead electricity prices.
Intel inks agreement with Sandia National Laboratories to explore neuromorphic computing
As a part of the U.S. Department of Energy's Advanced Scientific Computing Research program, Intel today inked a three-year agreement with Sandia National Laboratories to explore the value of neuromorphic computing for scaled-up AI problems. Sandia will kick off its work using a 50-million-neuron Loihi-based system recently delivered to its facility in Albuquerque, New Mexico. As the collaboration progresses, Intel says the labs will receive systems built on the company's next-generation neuromorphic architecture. Along with Intel, researchers at IBM, HP, MIT, Purdue, and Stanford hope to leverage neuromorphic computing -- circuits that mimic the nervous system's biology -- to develop supercomputers 1,000 times more powerful than any today. Chips like Loihi excel at constraint satisfaction problems, which require evaluating a large number of potential solutions to identify the one or few that satisfy specific constraints.
Tiny Machine Learning: The Next AI Revolution
Over the past decade, we have witnessed the size of machine learning algorithms grow exponentially due to improvements in processor speeds and the advent of big data. Initially, models were small enough to run on local machines using one or more cores within the central processing unit (CPU). Shortly after, computation using graphics processing units (GPUs) became necessary to handle larger datasets and became more readily available due to introduction of cloud-based services such as SaaS platforms (e.g., Google Colaboratory) and IaaS (e.g., Amazon EC2 Instances). At this time, algorithms could still be run on single machines. More recently, we have seen the development of specialized application-specific integrated circuits (ASICs) tensor processing units (TPUs) which can pack the power of 8 GPUs.