We propose a framework called HyperVAE for encoding distributions of distributions. When a target distribution is modeled by a VAE, its neural network parameters \theta is drawn from a distribution p(\theta) which is modeled by a hyper-level VAE. We propose a variational inference using Gaussian mixture models to implicitly encode the parameters \theta into a low dimensional Gaussian distribution. Given a target distribution, we predict the posterior distribution of the latent code, then use a matrix-network decoder to generate a posterior distribution q(\theta). HyperVAE can encode the parameters \theta in full in contrast to common hyper-networks practices, which generate only the scale and bias vectors as target-network parameters. Thus HyperVAE preserves much more information about the model for each task in the latent space. We discuss HyperVAE using the minimum description length (MDL) principle and show that it helps HyperVAE to generalize. We evaluate HyperVAE in density estimation tasks, outlier detection and discovery of novel design classes, demonstrating its efficacy.

Murdoch and Szlam Understanding and explaining decisions of complex (2017) explain long short term memory networks models such as neural networks has recently (LSTMs) (Hochreiter and Schmidhuber, 1997) by gained a lot of attention for engendering trust in means of ngram rules, but their rules are limited these models, improving them, and understanding to presence of single ngrams and do not capture them better (Montavon et al., 2018; Alishahi et al., interaction between ngrams in text. To learn explanation 2019; Belinkov and Glass, 2019). Several previous rules for RNNs while overcoming the studies developing interpretability techniques provide limitations of the previous approaches, we have the a set of input features or segments that are the following contributions in the paper: most salient for the model output. Approaches such as input perturbation and gradient computation are 1. We induce explanation rules over important popular for this purpose (Ancona et al., 2018; Arras skipgrams in text, while ensuring that these et al., 2019). A drawback of these approaches rules generalize to unseen data. To this end, is the lack of information about interaction between we quantify skipgram importance in LSTMs different features. While heatmaps (Li et al., by first pooling gradients across embedding 2016b,a; Arras et al., 2017) and partial dependence dimensions to compute word importance, and plots (Lundberg and Lee, 2017) are popularly used, thereby aggregating them into skipgram importance. Research conducted while at CLiPS.

We investigate meta-learning procedures in the setting of stochastic linear bandits tasks. The goal is to select a learning algorithm which works well on average over a class of bandits tasks, that are sampled from a task-distribution. Inspired by recent work on learning-to-learn linear regression, we consider a class of bandit algorithms that implement a regularized version of the well-known OFUL algorithm, where the regularization is a square euclidean distance to a bias vector. We first study the benefit of the biased OFUL algorithm in terms of regret minimization. We then propose two strategies to estimate the bias within the learning-to-learn setting. We show both theoretically and experimentally, that when the number of tasks grows and the variance of the task-distribution is small, our strategies have a significant advantage over learning the tasks in isolation.

Predictive services nowadays play an important role across all business sectors. However, deployed machine learning models are challenged by changing data streams over time which is described as concept drift. Prediction quality of models can be largely influenced by this phenomenon. Therefore, concept drift is usually handled by retraining of the model. However, current research lacks a recommendation which data should be selected for the retraining of the machine learning model. Therefore, we systematically analyze different data selection strategies in this work. Subsequently, we instantiate our findings on a use case in process mining which is strongly affected by concept drift. We can show that we can improve accuracy from 0.5400 to 0.7010 with concept drift handling. Furthermore, we depict the effects of the different data selection strategies.

Transfer learning, or domain adaptation, is concerned with machine learning problems in which training and testing data come from possibly different distributions (denoted as $\mu$ and $\mu'$, respectively). In this work, we give an information-theoretic analysis on the generalization error and the excess risk of transfer learning algorithms, following a line of work initiated by Russo and Zhou. Our results suggest, perhaps as expected, that the Kullback-Leibler (KL) divergence $D(mu||mu')$ plays an important role in characterizing the generalization error in the settings of domain adaptation. Specifically, we provide generalization error upper bounds for general transfer learning algorithms and extend the results to a specific empirical risk minimization (ERM) algorithm where data from both distributions are available in the training phase. We further apply the method to iterative, noisy gradient descent algorithms, and obtain upper bounds which can be easily calculated, only using parameters from the learning algorithms. A few illustrative examples are provided to demonstrate the usefulness of the results. In particular, our bound is tighter in specific classification problems than the bound derived using Rademacher complexity.

Author Neil Gaiman has admitted breaking Scotland's lockdown rules by travelling 11,000 miles from New Zealand to his holiday home on Skye. The Good Omens and American Gods writer left his wife and son in Auckland so he could "isolate" at his island retreat. He wrote on his online bog: "Hullo from Scotland, where I am in rural lockdown on my own." The science fiction and fantasy author has since been criticised for "endangering" local people". The SNP's Westminster leader Ian Blackford, who is the MP for the island, told the Sunday Times the author's journey was unacceptable. He said: "What is it about people, when they know we are in the middle of lockdown that they think they can come here from the other side of the planet, in turn endangering local people from exposure to this infection that they could have picked up at any step of the way?" Mr Gaiman - whose main family home is in Woodstock in the USA - has owned the house on Skye for more than 10 years. The English-born author wrote on his blog that until two weeks ago he had been living in New Zealand with his wife, the singer Amanda Palmer, and their four-year-old son. He said the couple agreed "that we needed to give each other some space". The 59-year-old said he flew "masked and gloved, from empty Auckland airport" to Los Angeles. He then caught a British Airways flight to London before borrowing a friend's car and heading for Skye. "I drove north, on empty motorways and then on empty roads, and got in about midnight, and I've been here ever since," he said. "I needed to be somewhere I could talk to people in the UK while they and I were awake, not just before breakfast and after dinner.

Like crystal balls for the universe's deeper mysteries, galaxies and other massive space objects can serve as lenses to more distant objects and phenomena along the same path, bending light in revelatory ways. Gravitational lensing was first theorized by Albert Einstein more than 100 years ago to describe how light bends when it travels past massive objects like galaxies and galaxy clusters. These lensing effects are typically described as weak or strong, and the strength of a lens relates to an object's position and mass and distance from the light source that is lensed. Strong lenses can have 100 billion times more mass than our sun, causing light from more distant objects in the same path to magnify and split, for example, into multiple images, or to appear as dramatic arcs or rings. The major limitation of strong gravitational lenses has been their scarcity, with only several hundred confirmed since the first observation in 1979, but that's changing, and fast.

Machine learning (ML) is the field of training machines to achieve high level of cognition and perform human-like analysis. Since ML is a data-driven approach, it seemingly fits into our daily lives and operations as well as complex and interdisciplinary fields. With the rise of commercial, open-source and user-catered ML tools, a key question often arises whenever ML is applied to explore a phenomenon or a scenario: what constitutes a good ML model? Keeping in mind that a proper answer to this question depends on a variety of factors, this work presumes that a good ML model is one that optimally performs and best describes the phenomenon on hand. From this perspective, identifying proper assessment metrics to evaluate performance of ML models is not only necessary but is also warranted. As such, this paper examines a number of the most commonly-used performance fitness and error metrics for regression and classification algorithms, with emphasis on engineering applications.

Brain gender differences have been known for a long time and are the possible reason for many psychological, psychiatric and behavioral differences between males and females. Predicting genders from brain functional connectivity (FC) can build the relationship between brain activities and gender, and extracting important gender related FC features from the prediction model offers a way to investigate the brain gender difference. Current predictive models applied to gender prediction demonstrate good accuracies, but usually extract individual functional connections instead of connectivity patterns in the whole connectivity matrix as features. In addition, current models often omit the effect of the input brain FC scale on prediction and cannot give any model uncertainty information. Hence, in this study we propose to predict gender from multiple scales of brain FC with deep learning, which can extract full FC patterns as features. We further develop the understanding of the feature extraction mechanism in deep neural network (DNN) and propose a DNN feature ranking method to extract the highly important features based on their contributions to the prediction. Moreover, we apply Bayesian deep learning to the brain FC gender prediction, which as a probabilistic model can not only make accurate predictions but also generate model uncertainty for each prediction. Experiments were done on the high-quality Human Connectome Project S1200 release dataset comprising the resting state functional MRI data of 1003 healthy adults. First, DNN reaches 83.0%, 87.6%, 92.0%, 93.5% and 94.1% accuracies respectively with the FC input derived from 25, 50, 100, 200, 300 independent component analysis (ICA) components. DNN outperforms the conventional machine learning methods on the 25-ICA-component scale FC, but the linear machine learning method catches up as the number of ICA components increases...

Stop-and-go traffic poses many challenges to tranportation system, but its formation and mechanism are still under exploration.however, it has been proved that by introducing Connected Automated Vehicles(CAVs) with carefully designed controllers one could dampen the stop-and-go waves in the vehicle fleet. Instead of using analytical model, this study adopts reinforcement learning to control the behavior of CAV and put a single CAV at the 2nd position of a vehicle fleet with the purpose to dampen the speed oscillation from the fleet leader and help following human drivers adopt more smooth driving behavior. The result show that our controller could decrease the spped oscillation of the CAV by 54% and 8%-28% for those following human-driven vehicles. Significant fuel consumption savings are also observed. Additionally, the result suggest that CAVs may act as a traffic stabilizer if they choose to behave slightly altruistically.