Coherent Gradients is a recently proposed hypothesis to explain why over-parameterized neural networks trained with gradient descent generalize well even though they have sufficient capacity to memorize the training set. Inspired by random forests, Coherent Gradients proposes that (Stochastic) Gradient Descent (SGD) finds common patterns amongst examples (if such common patterns exist) since descent directions that are common to many examples add up in the overall gradient, and thus the biggest changes to the network parameters are those that simultaneously help many examples. The original Coherent Gradients paper validated the theory through causal intervention experiments on shallow, fully connected networks on MNIST. In this work, we perform similar intervention experiments on more complex architectures (such as VGG, Inception and ResNet) on more complex datasets (such as CIFAR-10 and ImageNet). Our results are in good agreement with the small scale study in the original paper, thus providing the first validation of coherent gradients in more practically relevant settings. We also confirm in these settings that suppressing incoherent updates by natural modifications to SGD can significantly reduce overfitting--lending credence to the hypothesis that memorization occurs when few examples are responsible for most of the gradient used in the update. Furthermore, we use the coherent gradients theory to explore a new characterization of why some examples are learned earlier than other examples, i.e., "easy" and "hard" examples.

The quality of the generative models (Generative adversarial networks, Variational Auto-Encoders, ...) depends heavily on the choice of a good probability distance. However some popular metrics lack convenient properties such as (geodesic) convexity, fast evaluation and so on. To address these shortcomings, we introduce a class of distances that have built-in convexity. We investigate the relationship with some known paradigms (sliced distances, reproducing kernel Hilbert spaces, energy distances).The distances are shown to posses fast implementations andare included in an adapted Variational Auto-Encoder termed X-ray Sobolev Variational Auto-Encoder (XS-VAE) which produces good quality resultson standard generative datasets.

A new collaborative learning, called split learning, was recently introduced, aiming to protect user data privacy without revealing raw input data to a server. It collaboratively runs a deep neural network model where the model is split into two parts, one for the client and the other for the server. Therefore, the server has no direct access to raw data processed at the client. Until now, the split learning is believed to be a promising approach to protect the client's raw data; for example, the client's data was protected in healthcare image applications using 2D convolutional neural network (CNN) models. However, it is still unclear whether the split learning can be applied to other deep learning models, in particular, 1D CNN. In this paper, we examine whether split learning can be used to perform privacy-preserving training for 1D CNN models. To answer this, we first design and implement an 1D CNN model under split learning and validate its efficacy in detecting heart abnormalities using medical ECG data. We observed that the 1D CNN model under split learning can achieve the same accuracy of 98.9\% like the original (non-split) model. However, our evaluation demonstrates that split learning may fail to protect the raw data privacy on 1D CNN models. To address the observed privacy leakage in split learning, we adopt two privacy leakage mitigation techniques: 1) adding more hidden layers to the client side and 2) applying differential privacy. Although those mitigation techniques are helpful in reducing privacy leakage, they have a significant impact on model accuracy. Hence, based on those results, we conclude that split learning alone would not be sufficient to maintain the confidentiality of raw sequential data in 1D CNN models.

Supervised machine learning often requires human-annotated data. While annotator disagreement is typically interpreted as evidence of noise, population-level label distribution learning (PLDL) treats the collection of annotations for each data item as a sample of the opinions of a population of human annotators, among whom disagreement may be proper and expected, even with no noise present. From this perspective, a typical training set may contain a large number of very small-sized samples, one for each data item, none of which, by itself, is large enough to be considered representative of the underlying population's beliefs about that item. We propose an algorithmic framework and new statistical tests for PLDL that account for sampling size. We apply them to previously proposed methods for sharing labels across similar data items. We also propose new approaches for label sharing, which we call neighborhood-based pooling.

The goal of this challenge is to test the potential of Reinforcement Learning (RL) to control electrical power transmission, in the most cost-effective manner, while keeping people and equipment safe from harm. Solving this challenge may have very positive impacts on society, as governments move to decarbonize the electricity sector and to electrify other sectors, to help reach IPCC climate goals. Existing software, computational methods and optimal powerflow solvers are not adequate for real-time network operations on short temporal horizons in a reasonable computational time. With recent changes in electricity generation and consumption patterns, system operation is moving to become more of a stochastic rather than a deterministic control problem. In order to overcome these complexities, new computational methods are required. The intention of this challenge is to explore RL as a solution method for electricity network control. There may be under-utilized, cost-effective flexibility in the power network that RL techniques can identify and capitalize on, that human operators and traditional solution techniques are unaware of or unaccustomed to. An RL agent that can act in conjunction, or in parallel with human network operators, will optimize grid security and reliability, allowing more renewable resources to be connected while minimizing the cost and maintaining supply to customers, and preventing damage to electrical equipment. Another aim of the project is to broaden the audience for the problem of electricity network control and to foster collaboration between experts in both the power systems community and the wider RL/ML community.

Hot stellar systems (HSS) are a collection of stars bound together by gravitational attraction. These systems hold clues to many mysteries of outer space so understanding their origin, evolution and physical properties is important but remains a huge challenge. We used multivariate $t$-mixtures model-based clustering to analyze 13456 hot stellar systems from Misgeld & Hilker (2011) that included 12763 candidate globular clusters and found eight homogeneous groups using the Bayesian Information Criterion (BIC). A nonparametric bootstrap procedure was used to estimate the confidence of each of our clustering assignments. The eight obtained groups can be characterized in terms of the correlation, mass, effective radius and surface density. Using conventional correlation-mass-effective radius-surface density notation, the largest group, Group 1, can be described as having positive-low-low-moderate characteristics. The other groups, numbered in decreasing sizes are similarly characterised, with Group 2 having positive-low-low-high characteristics, Group 3 displaying positive-low-low-moderate characteristics, Group 4 having positive-low-low-high characteristic, Group 5 displaying positive-low-moderate-moderate characteristic and Group 6 showing positive-moderate-low-high characteristic. The smallest group (Group 8) shows negative-low-moderate-moderate characteristic. Group 7 has no candidate clusters and so cannot be similarly labeled but the mass, effective radius correlation for these non-candidates indicates that they zare larger than typical globular clusters. Assertions drawn for each group are ambiguous for a few HSS having low confidence in classification. Our analysis identifies distinct kinds of HSS with varying confidence and provides novel insight into their physical and evolutionary properties.

Reinforcement learning systems require good representations to work well. For decades practical success in reinforcement learning was limited to small domains. Deep reinforcement learning systems, on the other hand, are scalable, not dependent on domain specific prior knowledge and have been successfully used to play Atari, in 3D navigation from pixels, and to control high degree of freedom robots. Unfortunately, the performance of deep reinforcement learning systems is sensitive to hyper-parameter settings and architecture choices. Even well tuned systems exhibit significant instability both within a trial and across experiment replications. In practice, significant expertise and trial and error are usually required to achieve good performance. One potential source of the problem is known as catastrophic interference: when later training decreases performance by overriding previous learning. Interestingly, the powerful generalization that makes Neural Networks (NN) so effective in batch supervised learning might explain the challenges when applying them in reinforcement learning tasks. In this paper, we explore how online NN training and interference interact in reinforcement learning. We find that simply re-mapping the input observations to a high-dimensional space improves learning speed and parameter sensitivity. We also show this preprocessing reduces interference in prediction tasks. More practically, we provide a simple approach to NN training that is easy to implement, and requires little additional computation. We demonstrate that our approach improves performance in both prediction and control with an extensive batch of experiments in classic control domains.

In February 2020, Robotics Business Review followed a total of 25 reported investments, mergers and acquisitions, and other transactions around robots, autonomous vehicles, drones, and related technologies. In addition to the usual autonomous vehicle and healthcare robotics companies, field robotics raised funding in February 2020. The reported values were worth a total of $1.19 billion. In comparison, Robotics Business Review and sibling site The Robot Report recorded $1.16 billion in transactions in January 2020, and $4.3 billion in February 2019. The number of investments declined from 40 last month and 25 a year ago.

The coronavirus disease COVID-19 has so far caused about 3,400 deaths, infected about 100.000 people, and is significantly impacting the economy in many countries. We used predictive analytics, a branch of artificial intelligence (AI), to forecast how many confirmed COVID-19 cases and deaths can be expected in the near future. Our method predicts that by March 13, the virus death toll will have climbed to 3,913, and confirmed cases will reach 116,250 worldwide, based on data available up to March 5. To develop contingency plans and hopefully head off the worst effects of the coronavirus, governments need to be able to anticipate the future course of the outbreak. This is where predictive analytics could prove invaluable.

Over the past few years, artificial intelligence has matured into a collection of powerful technologies that are delivering competitive advantage to businesses across industries. Global AI adoption and investment are soaring. By one account, 37 percent of organizations have deployed AI solutions--up 270 percent from four years ago.1 Analysts forecast global AI spending will more than double over the next three years, topping US$79 billion by 2022.2 Get the Deloitte Insights app. Deloitte's State of AI in the Enterprise, 2nd Edition offers a global perspective of AI early adopters, based on surveying 1,900 IT and business executives from seven countries and a variety of industries.3 These adopters are increasing their spending on AI technologies and realizing positive returns. Almost two-thirds (65 percent) report that AI technologies are enabling their organizations to move ahead of the competition.