One of the generally accepted views of modern deep learning is that increasing the number of parameters usually leads to better quality. The two easiest ways to increase the number of parameters is to increase the size of the network, e.g. width, or to train a deep ensemble; both approaches improve the performance in practice. In this work, we consider a fixed memory budget setting, and investigate, what is more effective: to train a single wide network, or to perform a memory split -- to train an ensemble of several thinner networks, with the same total number of parameters? We find that, for large enough budgets, the number of networks in the ensemble, corresponding to the optimal memory split, is usually larger than one. Interestingly, this effect holds for the commonly used sizes of the standard architectures. For example, one WideResNet-28-10 achieves significantly worse test accuracy on CIFAR-100 than an ensemble of sixteen thinner WideResNets: 80.6% and 82.52% correspondingly. We call the described effect the Memory Split Advantage and show that it holds for a variety of datasets and model architectures.

Large-scale population-based studies in medicine are a key resource towards better diagnosis, monitoring, and treatment of diseases. They also serve as enablers of clinical decision support systems, in particular Computer Aided Diagnosis (CADx) using machine learning (ML). Numerous ML approaches for CADx have been proposed in literature. However, these approaches assume full data availability, which is not always feasible in clinical data. To account for missing data, incomplete data samples are either removed or imputed, which could lead to data bias and may negatively affect classification performance. As a solution, we propose an end-to-end learning of imputation and disease prediction of incomplete medical datasets via Multigraph Geometric Matrix Completion (MGMC). MGMC uses multiple recurrent graph convolutional networks, where each graph represents an independent population model based on a key clinical meta-feature like age, sex, or cognitive function. Graph signal aggregation from local patient neighborhoods, combined with multigraph signal fusion via self-attention, has a regularizing effect on both matrix reconstruction and classification performance. Our proposed approach is able to impute class relevant features as well as perform accurate classification on two publicly available medical datasets. We empirically show the superiority of our proposed approach in terms of classification and imputation performance when compared with state-of-the-art approaches. MGMC enables disease prediction in multimodal and incomplete medical datasets. These findings could serve as baseline for future CADx approaches which utilize incomplete datasets.

Echo State Networks (ESNs) are a class of single-layer recurrent neural networks with randomly generated internal weights, and a single layer of tuneable outer weights, which are usually trained by regularised linear least squares regression. Remarkably, ESNs still enjoy the universal approximation property despite the training procedure being entirely linear. In this paper, we prove that an ESN trained on a sequence of scalar observations from an ergodic dynamical system (with invariant measure {\mu}) using Tikhonov least squares will approximate future observations of the dynamical system in the L2({\mu}) norm. We call this the ESN Training Theorem. We demonstrate the theory numerically by training an ESN using Tikhonov least squares on a sequence of scalar observations of the Lorenz system, and compare the invariant measure of these observations with the invariant measure of the future predictions of the autonomous ESN.

We propose a novel activity learning framework based on Edge Cloud architecture for the purpose of recognizing and predicting human activities. Although activity recognition has been vastly studied by many researchers, the temporal features that constitute an activity, which can provide useful insights for activity models, have not been exploited to their full potentials by mining algorithms. In this paper, we utilize temporal features for activity recognition and prediction in a single smart home setting. We discover activity patterns and temporal relations such as the order of activities from real data to develop a prompting system. Analysis of real data collected from smart homes was used to validate the proposed method.

Adversarial attacks against conventional Deep Learning (DL) systems and algorithms have been widely studied, and various defenses were proposed. However, the possibility and feasibility of such attacks against Deep Reinforcement Learning (DRL) are less explored. As DRL has achieved great success in various complex tasks, designing effective adversarial attacks is an indispensable prerequisite towards building robust DRL algorithms. In this paper, we introduce two novel adversarial attack techniques to \emph{stealthily} and \emph{efficiently} attack the DRL agents. These two techniques enable an adversary to inject adversarial samples in a minimal set of critical moments while causing the most severe damage to the agent. The first technique is the \emph{critical point attack}: the adversary builds a model to predict the future environmental states and agent's actions, assesses the damage of each possible attack strategy, and selects the optimal one. The second technique is the \emph{antagonist attack}: the adversary automatically learns a domain-agnostic model to discover the critical moments of attacking the agent in an episode. Experimental results demonstrate the effectiveness of our techniques. Specifically, to successfully attack the DRL agent, our critical point technique only requires 1 (TORCS) or 2 (Atari Pong and Breakout) steps, and the antagonist technique needs fewer than 5 steps (4 Mujoco tasks), which are significant improvements over state-of-the-art methods.

In this work, we aim at making agents communicate On the other hand, multi-agent communication with humans in natural language. Our starting research (Foerster et al., 2016; Lazaridou et al., point is a language model that has been trained on 2017; Havrylov and Titov, 2017; Evtimova et al., generic, not task-specific language data. We then 2017; Lee et al., 2019) puts communication at the place this model in a multi-agent communication heart of agents' (language) learning. Implemented environment that generates task-specific rewards, within a multi-agent reinforcement learning setup, which are used to adapt or modulate the model, agents start tabula rasa and form communication making it task-conditional. We thus propose to decompose protocols that maximize task rewards. While this the problem of learning language use into purely utilitarian framework results in agents that two components: learning "what" to say based on successfully learn to solve the task by creating a a given situation, and learning "how" to say it. The communication protocol, these emergent communication "what" is the essence of communication that underlies protocols do not bear core properties of our intentions and is chosen by maximizing any natural language. Chaabouni et al. (2019) show that given utility, making it a functional, utility-driven protocols found through emergent communication, process. On the other hand, the "how" is a surface unlike natural language, do not conform to Zipf's realization of our intentions, i.e., the words we use Law of Abbreviation; Kottur et al. (2017) find that

We present a self-learning approach that combines deep reinforcement learning and Monte Carlo tree search to solve the traveling salesman problem. The proposed approach has two advantages. First, it adopts deep reinforcement learning to compute the value functions for decision, which removes the need of hand-crafted features and labelled data. Second, it uses Monte Carlo tree search to select the best policy by comparing different value functions, which increases its generalization ability. Experimental results show that the proposed method performs favorably against other methods in small-to-medium problem settings. And it shows comparable performance as state-of-the-art in large problem setting.

The semiconductor industry as we know it is facing a critical roadblock that will lead to the end of Moore's law. As transistors continue to shrink, quantum effects have a significant negative consequence on their operation. As such, the development of "beyond CMOS devices" has begun. The push for devices that are cheaper, smaller, and faster has led to the use of scanning probe fabrication. One of the first examples of such a technique was IBM's video "A boy and his atom", where CO molecules were moved along a Cu surface using a sharp metallic tip.

NEW YORK, NY / ACCESSWIRE / May 13, 2020 /Over the past decade, machine learning, particularly deep learning methods based on artificial neural networks, has made a number of remarkable advances that have improved our ability to build more accurate systems in a wide range of fields, including computer vision, speech recognition, language translation, and natural language understanding tasks.Today, with the emergence of GPU, FPGA, XPU and other AI computing servers, the computing power of AI has been greatly improved, and the selection of algorithm framework has become an important factor to optimize the computing efficiency of AI. At the same time, due to the development of AI computing system from single-machine single-card to single-machine multi-card, and then to multi-machine multi-card parallel computing, data centers need to simultaneously manage a large number of AI computing servers to support applications. How to better manage and monitor will also affect the output efficiency and operation cost of AI application.Machine learning, particularly deep learning, is driving the development of artificial intelligence (AI). The need for more efficient hardware acceleration of AI/ML/DL is recognized in academia and industry. This year, we've seen a growing number of players, including some of the world's top semiconductor …

At first sight, these kinds of examples may seem amusing but less interesting, and irrelevant to deploying agents in the real world, where there are no simulator bugs. However, the underlying problem isn't the bug itself but a failure of abstraction that can be exploited by the agent. In the example above, the robot's task was misspecified because of incorrect assumptions about simulator physics. Analogously, a real-world traffic optimisation task might be misspecified by incorrectly assuming that the traffic routing infrastructure does not have software bugs or security vulnerabilities that a sufficiently clever agent could discover. Such assumptions need not be made explicitly – more likely, they are details that simply never occurred to the designer.