Driver identification has emerged as a vital research field, where both practitioners and researchers investigate the potential of driver identification to enable a personalized driving experience. Within recent years, a selection of studies have reported that individuals could be perfectly identified based on their driving behavior under controlled conditions. However, research investigating the potential of driver identification under naturalistic conditions claim accuracies only marginally higher than random guess. The paper at hand provides a comprehensive summary of the recent work, highlighting the main discrepancies in the design of the machine learning approaches, primarily the window length parameter that was considered. Key findings further indicate that the longitudinal vehicle control information is particularly useful for driver identification, leaving the research gap on the extent to which the lateral vehicle control can be used for reliable identification. Building upon existing work, we provide a novel approach for the design of the window length parameter that provides evidence that reliable driver identification can be achieved with data limited to the steering wheel only. The results and insights in this paper are based on data collected from the largest naturalistic driving study conducted in this field. Overall, a neural network based on GRUs was found to provide better identification performance than traditional methods, increasing the prediction accuracy from under 15\% to over 65\% for 15 drivers. When leveraging the full field study dataset, comprising 72 drivers, the accuracy of identification prediction of the approach improved a random guess approach by a factor of 25.

Learning the parameters of a Gaussian mixtures models is a fundamental and widely studied problem with numerous applications. In this work, we give new algorithms for learning the parameters of a high-dimensional, well separated, Gaussian mixture model subject to the strong constraint of differential privacy. In particular, we give a differentially private analogue of the algorithm of Achlioptas and McSherry. Our algorithm has two key properties not achieved by prior work: (1) The algorithm's sample complexity matches that of the corresponding non-private algorithm up to lower order terms in a wide range of parameters. (2) The algorithm does not require strong a priori bounds on the parameters of the mixture components.

Real Time Trajectory Prediction Using Deep Conditional Generative Models Sebastian Gomez-Gonzalez 1, 2, Sergey Prokudin 1, Bernhard Sch olkopf 1 and Jan Peters 2 Abstract -- Data driven methods for time series forecasting that quantify uncertainty open new important possibilities for robot tasks with hard real time constraints, allowing the robot system to make decisions that trade off between reaction time and accuracy in the predictions. Despite the recent advances in deep learning, it is still challenging to make long term accurate predictions with the low latency required by real time robotic systems. In this paper, we propose a deep conditional generative model for trajectory prediction that is learned from a data set of collected trajectories. Our method uses an encoder and decoder deep networks that maps complete or partial trajectories to a Gaussian distributed latent space and back, allowing for fast inference of the future values of a trajectory given previous observations. The encoder and decoder networks are trained using stochastic gradient variational Bayes. In the experiments, we show that our model provides more accurate long term predictions with a lower latency that popular models for trajectory forecasting like recurrent neural networks or physical models based on differential equations. Finally, we test our proposed approach in a robot table tennis scenario to evaluate the performance of the proposed method in a robotic task with hard real time constraints. I. INTRODUCTION Dynamic high speed robotics tasks often require accurate methods to forecast the future value of a physical quantity based on previous measurements while respecting the real time constraints of the particular application.

A new meta-algorithm for estimating the conditional average treatment effects is proposed in the paper. The main idea underlying the algorithm is to consider a new dataset consisting of feature vectors produced by means of concatenation of examples from control and treatment groups, which are close to each other. Outcomes of new data are defined as the difference between outcomes of the corresponding examples comprising new feature vectors. The second idea is based on the assumption that the number of controls is rather large and the control outcome function is precisely determined. This assumption allows us to augment treatments by generating feature vectors which are closed to available treatments. The outcome regression function constructed on the augmented set of concatenated feature vectors can be viewed as an estimator of the conditional average treatment effects. A simple modification of the Co-learner based on the random subspace method or the feature bagging is also proposed. Various numerical simulation experiments illustrate the proposed algorithm and show its outperformance in comparison with the well-known T-learner and X-learner for several types of the control and treatment outcome functions. Keywords: treatment effect, meta-learner, regression, treatment, control, simulation 1 Introduction One the most important problems in medicine is to choose the most appropriate treatment for a certain patient which may differ from other patients in her/his clinical or other characteristics [25]. With the increase of the amount of data and with the developing the electronic health record concept in medicine, there is a growing interest to apply machine learning methods to solve the problem of the most appropriate treatment by estimating treatment effects directly from observational data. The main peculiarity of observational data is that it contains past actions, their outcomes, but without direct access to the mechanism which gave rise to the action. Shalit at al. [34] give a clear example of observational data, when we have patient characteristics, medications (action), and outcomes, 1 arXiv:1909.03894v1

We present Graph Random Neural Features (GRNF), a novel embedding method from graph-structured data to real vectors based on a family of graph neural networks. The embedding naturally deals with graph isomorphism and preserves, in probability, the metric structure of graph domain. In addition to being an explicit embedding method, it also allows to efficiently and effectively approximate graph metric distances (as well as complete kernel functions); a criterion to select the embedding dimension trading off the approximation accuracy with the computational cost is also provided. Derived GRNF can be used within traditional processing methods or as input layer of a larger graph neural network. The theoretical guarantees that accompany GRNF ensure that the considered graph distance is metric, hence allowing to distinguish any pair of non-isomorphic graphs. 1 Introduction Inference on graph-structured data is one of the hottest topics in machine learning, thanks to successes achieved in several scientific fields, like neurosciences, chemistry, computational biology and social sciences [1-3]. One of the major research challenges there consists of building a practical solution able to process graphs, yet managing the graph isomorphism problem. A way to address this latter problem passes through metric distances and complete kernels, however, it has been shown to be at least as hard as deciding whether two graphs are isomorphic [4].

In this work we systematically analyze general properties of differential equations used as machine learning models. We demonstrate that the gradient of the loss function with respect to to the hidden state can be considered as a generalized momentum conjugate to the hidden state, allowing application of the tools of classical mechanics. In addition, we show that not only residual networks, but also feedforward neural networks with small nonlinearities and the weights matrices deviating only slightly from identity matrices can be related to the differential equations. We propose a differential equation describing such networks and investigate its properties. 1 Introduction Deep learning is a form of machine learning that uses neural networks with many hidden layers [1, 2]. Deep learning models have dramatically improved speech recognition, visual object recognition, object detection and many other domains [2]. Since the number of layers in deep neural networks become large, it is possible to consider the layer number as a continuous variable [3] and represent the neural network by an differential equation. The connection between the neural networks and differential equations first appeared with an additive model for continuous time recurrent neural networks, described by the differential equations [4] τ i dx i dt x i n null j 1w j,iσ ( x j θ j) I i(t) . Hopfield's work [6] pioneered the analog computation of continuous time recurrent neural networks instead of digital computation using complex numerical algorithms on a digital computer. A Hopfield network has a quadratic form as an Lyapunov function for the activity dynamics. As a consequence, the state of the network evolves to a final state that is a minimum of the Lyapunov function when started in any initial state [7].

Continual learning of deep neural networks is a key requirement for scaling them up to more complex applicative scenarios and for achieving real lifelong learning of these architectures. Previous approaches to the problem have considered either the progressive increase in the size of the networks, or have tried to regularize the network behavior to equalize it with respect to previously observed tasks. In the latter case, it is essential to understand what type of information best represents this past behavior. Common techniques include regularizing the past outputs, gradients, or individual weights. In this work, we propose a new, relatively simple and efficient method to perform continual learning by regularizing instead the network internal embeddings. To make the approach scalable, we also propose a dynamic sampling strategy to reduce the memory footprint of the required external storage. We show that our method performs favorably with respect to state-of-the-art approaches in the literature, while requiring significantly less space in memory and computational time. In addition, inspired inspired by to recent works, we evaluate the impact of selecting a more flexible model for the activation functions inside the network, evaluating the impact of catastrophic forgetting on the activation functions themselves.

Machine Learning (ML) is proving extremely beneficial in many healthcare applications. In pediatric oncology, retrospective studies that investigate the relationship between treatment and late adverse effects still rely on simple heuristics. To assess the effects of radiation therapy, treatment plans are typically simulated on phantoms, i.e., virtual surrogates of patient anatomy. Currently, phantoms are built according to reasonable, yet simple, human-designed criteria. This often results in a lack of individualization. We present a novel approach that combines imaging and ML to build individualized phantoms automatically. Given the features of a patient treated historically (only 2D radiographs available), and a database of 3D Computed Tomography (CT) imaging with organ segmentations and relative patient features, our approach uses ML to predict how to assemble a patient-specific phantom automatically. Experiments on 60 abdominal CTs of pediatric patients show that our approach constructs significantly more representative phantoms than using current phantom building criteria, in terms of location and shape of the abdomen and of two considered organs, the liver and the spleen. Among several ML algorithms considered, the Gene-pool Optimal Mixing Evolutionary Algorithm for Genetic Programming (GP-GOMEA) is found to deliver the best performing models, which are, moreover, transparent and interpretable mathematical expressions.

Granger causality analysis, as one of the most popular time series causality methods, has been widely used in the economics, neuroscience. However, unobserved confounders is a fundamental problem in the observational studies, which is still not solved for the non-linear Granger causality. The application works often deal with this problem in virtue of the proxy variables, who can be treated as a measure of the confounder with noise. But the proxy variables has been proved to be unreliable, because of the bias it may induce. In this paper, we try to "recover" the unobserved confounders for the Granger causality. We use a generative model with latent variable to build the relationship between the unobserved confounders and the observed variables(tested variable and the proxy variables). The posterior distribution of the latent variable is adopted to represent the confounders distribution, which can be sampled to get the estimated confounders. We adopt the variational autoencoder to estimate the intractable posterior distribution. The recurrent neural network is applied to build the temporal relationship in the data. We evaluate our method in the synthetic and semi-synthetic dataset. The result shows our estimated confounders has a better performance than the proxy variables in the non-linear Granger causality with multiple proxies in the semi-synthetic dataset. But the performances of the synthetic dataset and the different noise level of proxy seem terrible. Any advice can really help.

\textit{Objective}: Joint analysis of multi-subject brain imaging datasets has wide applications in biomedical engineering. In these datasets, some sources belong to all subjects (joint), a subset of subjects (partially-joint), or a single subject (individual). In this paper, this source model is referred to as joint/partially-joint/individual multiple datasets multidimensional (JpJI-MDM), and accordingly, a source extraction method is developed. Method: We present a deflation-based algorithm utilizing higher-order cumulants to analyze the JpJI-MDM source model. The algorithm maximizes a cost function which leads to an eigenvalue problem solved with a subspace iteration method. Furthermore, we introduce the JpJI-feature which indicates the spatial shape of each source and the amount of its jointness with other subjects. We use this feature to determine the type of sources. Results: We evaluate our algorithm by analyzing simulated data and two real functional magnetic resonance imaging (fMRI) datasets. In our simulation study, we will show that the proposed algorithm determines the type of sources with an accuracy of 95% and 100% for 2-class and 3-class clustering scenarios, respectively. Furthermore, our algorithm extracts meaningful joint and partially-joint sources from the two real datasets, which are consistent with the existing neuroscience studies. Conclusion: Our results in analyzing the real datasets reveal that both datasets follow the JpJI-MDM source model. This source model improves the accuracy of source extraction methods developed for multi-subject datasets. Significance: The proposed joint blind source separation algorithm is robust and avoids parameters which are difficult to fine-tune.