A core operation in reinforcement learning (RL) is finding an action that is optimal with respect to a learned state-action value function. This operation is often challenging when the learned value function takes continuous actions as input. We introduce deep RBF value functions: state-action value functions learned using a deep neural network with a radial-basis function (RBF) output layer. We show that the optimal action with respect to a deep RBF value function can be easily approximated up to any desired accuracy. Moreover, deep RBF value functions can represent any true value function up to any desired accuracy owing to their support for universal function approximation. By learning a deep RBF value function, we extend the standard DQN algorithm to continuous control, and demonstrate that the resultant agent, RBF-DQN, outperforms standard baselines on a set of continuous-action RL problems.

With the strengthening of data privacy and security, traditional data centralization for AI faces huge challenges. Moreover, isolated data existing in various industries and institutions is grossly underused and thus retards the advance of AI applications. We propose a possible solution to these problems: knowledge federation. Beyond the concepts of federated learning and secure multi-party computation, we introduce a comprehensive knowledge federation framework, which is a hierarchy with four-level federation. In terms of the occurrence time of federation, knowledge federation can be categorized into information level, model level, cognition level, and knowledge level. To facilitate widespread academic and commercial adoption of this concept, we provide definitions free from ambiguity for the knowledge federation framework. In addition, we clarify the relationship and differentiation between knowledge federation and other related research fields and conclude that knowledge federation is a unified framework for secure multi-party computation and learning.

This paper establishes a precise high-dimensional asymptotic theory for Boosting on separable data, taking statistical and computational perspectives. We consider the setting where the number of features (weak learners) p scales with the sample size n, in an over-parametrized regime. On the statistical front, we provide an exact analysis of the generalization error of Boosting, when the algorithm interpolates the training data and maximizes an empirical L1 margin. The angle between the Boosting solution and the ground truth is characterized explicitly. On the computational front, we provide a sharp analysis of the stopping time when Boosting approximately maximizes the empirical L1 margin. Furthermore, we discover that, the larger the margin, the smaller the proportion of active features (with zero initialization). At the heart of our theory lies a detailed study of the maximum L1 margin, using tools from convex geometry. The maximum L1 margin can be precisely described by a new system of non-linear equations, which we study using a novel uniform deviation argument. Preliminary numerical results are presented to demonstrate the accuracy of our theory.

In conventional supervised learning, a training dataset is given with ground-truth labels from a known label set, and the learned model will classify unseen instances to the known labels. In this paper, we study a new problem setting in which there are unknown classes in the training dataset misperceived as other labels, and thus their existence appears unknown from the given supervision. We attribute the unknown unknowns to the fact that the training dataset is badly advised by the incompletely perceived label space due to the insufficient feature information. To this end, we propose the exploratory machine learning, which examines and investigates the training dataset by actively augmenting the feature space to discover potentially unknown labels. Our approach consists of three ingredients including rejection model, feature acquisition, and model cascade. The effectiveness is validated on both synthetic and real datasets.

We present a self-supervised deep multi-task learning framework for electrocardiogram (ECG) -based emotion recognition. The proposed framework consists of two stages of learning a) learning ECG representations and b) learning to classify emotions. ECG representations are learned by a signal transformation recognition network. The network learns high-level abstract representations from unlabeled ECG data. Six different signal transformations are applied to the ECG signals, and transformation recognition is performed as pretext tasks. Training the model on pretext tasks helps our network learn spatiotemporal representations that generalize well across different datasets and different emotion categories. We transfer the weights of the self-supervised network to an emotion recognition network, where the convolutional layers are kept frozen and the dense layers are trained with labelled ECG data. We show that our proposed method considerably improves the performance compared to a network trained using fully-supervised learning. New state-of-the-art results are set in classification of arousal, valence, affective states, and stress for the four utilized datasets. Extensive experiments are performed, providing interesting insights into the impact of using a multi-task self-supervised structure instead of a single-task model, as well as the optimum level of difficulty required for the pretext self-supervised tasks.

Topology optimization by distributing materials in a domain requires stochastic optimizers to solve highly complicated problems. However, solving such problems requires millions of finite element calculations with hundreds of design variables or more involved , whose computational cost is huge and often unacceptable. To speed up computation, here we report a method to integrate deep learning into stochastic optimization algorithm. A Deep Neural Network (DNN) learns and substitutes the objective function by forming a loop with Generative Simulated Annealing (GSA). In each iteration, GSA uses DNN to evaluate the objective function to obtain an optimized solution, based on which new training data are generated; thus, DNN enhances its accuracy and GSA could accordingly improve its solution in next iteration until convergence. Our algorithm was tested by compliance minimization problems and reduced computational time by over two orders of magnitude. This approach sheds light on solving large multi-dimensional optimization problems.

Subsampled Randomized Hadamard Transform (SRHT), a popular random projection method that can efficiently project a $d$-dimensional data into $r$-dimensional space ($r \ll d$) in $O(dlog(d))$ time, has been widely used to address the challenge of high-dimensionality in machine learning. SRHT works by rotating the input data matrix $\mathbf{X} \in \mathbb{R}^{n \times d}$ by Randomized Walsh-Hadamard Transform followed with a subsequent uniform column sampling on the rotated matrix. Despite the advantages of SRHT, one limitation of SRHT is that it generates the new low-dimensional embedding without considering any specific properties of a given dataset. Therefore, this data-independent random projection method may result in inferior and unstable performance when used for a particular machine learning task, e.g., classification. To overcome this limitation, we analyze the effect of using SRHT for random projection in the context of linear SVM classification. Based on our analysis, we propose importance sampling and deterministic top-$r$ sampling to produce effective low-dimensional embedding instead of uniform sampling SRHT. In addition, we also proposed a new supervised non-uniform sampling method. Our experimental results have demonstrated that our proposed methods can achieve higher classification accuracies than SRHT and other random projection methods on six real-life datasets.

Classical methods for psychometric function estimation either require excessive measurements or produce only a low-resolution approximation of the target psychometric function. In this paper, we propose a novel solution for rapid screening for a change in the psychometric function estimation of a given patient. We use Bayesian active model selection to perform an automated pure-tone audiogram test with the goal of quickly finding if the current audiogram will be different from a previous audiogram. We validate our approach using audiometric data from the National Institute for Occupational Safety and Health NIOSH. Initial results show that with a few tones we can detect if the patient's audiometric function has changed between the two test sessions with high confidence.

Bayesian networks (BNs) have received increasing research attention that is not matched by adoption in practice and yet have potential to significantly benefit healthcare. Hitherto, research works have not investigated the types of medical conditions being modelled with BNs, nor whether any differences exist in how and why they are applied to different conditions. This research seeks to identify and quantify the range of medical conditions for which healthcare-related BN models have been proposed, and the differences in approach between the most common medical conditions to which they have been applied. We found that almost two-thirds of all healthcare BNs are focused on four conditions: cardiac, cancer, psychological and lung disorders. We believe that a lack of understanding regarding how BNs work and what they are capable of exists, and that it is only with greater understanding and promotion that we may ever realise the full potential of BNs to effect positive change in daily healthcare practice.

There has been much recent progress in forecasting the next observation of a linear dynamical system (LDS), which is known as the improper learning, as well as in the estimation of its system matrices, which is known as the proper learning of LDS. We present an approach to proper learning of LDS, which in spite of the non-convexity of the problem, guarantees global convergence of numerical solutions to a least-squares estimator. We present promising computational results.