The increasing availability of electrocardiogram (ECG) data has motivated the use of data-driven models for automating various clinical tasks based on ECG data. The development of subject-specific models are limited by the cost and difficulty of obtaining sufficient training data for each individual. The alternative of population model, however, faces challenges caused by the significant inter-subject variations within the ECG data. We address this challenge by investigating for the first time the problem of learning representations for clinically-informative variables while disentangling other factors of variations within the ECG data. In this work, we present a conditional variational autoencoder (VAE) to extract the subject-specific adjustment to the ECG data, conditioned on task-specific representations learned from a deterministic encoder. To encourage the representation for inter-subject variations to be independent from the task-specific representation, maximum mean discrepancy is used to match all the moments between the distributions learned by the VAE conditioning on the code from the deterministic encoder. The learning of the task-specific representation is regularized by a weak supervision in the form of contrastive regularization. We apply the proposed method to a novel yet important clinical task of classifying the origin of ventricular tachycardia (VT) into pre-defined segments, demonstrating the efficacy of the proposed method against the standard VAE.

Furthermore, traveling in any direction orthogonal to the gradient maintains the value of the function. In this work, we show that these orthogonal directions that are ignored by gradient descent can be critical in equilibrium problems. Equilibrium problems have drawn heightened attention in machine learning due to the emergence of the Generative Adversarial Network (GAN). We use the framework of Variational Inequalities to analyze popular training algorithms for a fundamental GAN variant: the Wasserstein Linear-Quadratic GAN. We show that the steepest descent direction causes divergence from the equilibrium, and guaranteed convergence to the equilibrium is achieved through following a particular orthogonal direction. We call this successful technique Crossing-the-Curl, named for its mathematical derivation as well as its intuition: identify the game's axis of rotation and move "across" space in the direction towards smaller "curling".

Given the number of people interested in my first post for visualizing Classification Models Results, I've decided to create and share some new function to visualize and compare whole Linear Regression Models with one line of code. These plots will help us with our time invested in model selection and a general understanding of our results. Where are we going with this post? Let's take a quick look at the final output: a quick nice dashboard with everything you'd need to compare and evaluate if your regression model is looking good, compare with others, or get working on further improvements. Interesting to say that, the exact same function mplot_full used before in the Part 1 – Classification Models post, will work on Regressions too lares::updateLares().

Valve's Team Fortress 2 (TF2) is a vastly popular online first-person shooter (FPS) with a fanbase and support that stems back a decade. It's rare that a FPS continues to carry such a strong community. However, it's fair to say that TF2 has continued to evolve as a platform, with significant changes to this weird and eclectic shooter since its original launch back in 2007. But it's not the game itself I'm interested in today -- given that I typically talk about the AI that works behind the scenes. Rather, I'm going to talk about at its community.

Multi-Task Gaussian processes (MTGPs) have shown a significant progress both in expressiveness and interpretation of the relatedness between different tasks: from linear combinations of independent single-output Gaussian processes (GPs), through the direct modeling of the cross-covariances such as spectral mixture kernels with phase shift, to the design of multivariate covariance functions based on spectral mixture kernels which model delays among tasks in addition to phase differences, and which provide a parametric interpretation of the relatedness across tasks. In this paper we further extend expressiveness and interpretability of MTGPs models and introduce a new family of kernels capable to model nonlinear correlations between tasks as well as dependencies between spectral mixtures, including time and phase delay. Specifically, we use generalized convolution spectral mixture kernels for modeling dependencies at spectral mixture level, and coupling coregionalization for discovering task level correlations. The proposed kernels for MTGP are validated on artificial data and compared with existing MTGPs methods on three real-world experiments. Results indicate the benefits of our more expressive representation with respect to performance and interpretability.

Internet of Things (IoT) is becoming truly ubiquitous in our everyday life, but it also faces unique security challenges. Intrusion detection is critical for the security and safety of a wireless IoT network. This paper discusses the human-in-the-loop active learning approach for wireless intrusion detection. We first present the fundamental challenges against the design of a successful Intrusion Detection System (IDS) for wireless IoT network. We then briefly review the rudimentary concepts of active learning and propose its employment in the diverse applications of wireless intrusion detection. Experimental example is also presented to show the significant performance improvement of the active learning method over traditional supervised learning approach. While machine learning techniques have been widely employed for intrusion detection, the application of human-in-the-loop machine learning that leverages both machine and human intelligence to intrusion detection of IoT is still in its infancy. We hope this article can assist the readers in understanding the key concepts of active learning and spur further research in this area.

Neural networks have many successful applications, while much less theoretical understanding has been gained. Towards bridging this gap, we study the problem of learning a two-layer overparameterized ReLU neural network for multi-class classification via stochastic gradient descent (SGD) from random initialization. In the overparameterized setting, when the data comes from mixtures of well-separated distributions, we prove that SGD learns a network with a small generalization error, albeit the network has enough capacity to fit arbitrary labels. Furthermore, the analysis provides interesting insights into several aspects of learning neural networks and can be verified based on empirical studies on synthetic data and on the MNIST dataset.

Python is a high-level language created by Guido van Rossum and first released in 1991. It is named after the greatest comedy act of all time, Monty Python. Python can be used to create pretty much any type of application. Python has been popular for many years and it's popularity shows no signs of stopping anytime soon. Being an in-demand language, knowing Python is beneficial for your career as a software developer. Having working knowledge of high-level programming languages is something any software developer should have. Here's a list of some of the best Python tutorials, Python courses, and Python books in 2018 to help you learn Python. Get the Ultimate Python Development kit. Get everything you need to code Python. Top package includes Egghead.io at the lowest price ever!

Given a set of human's decisions that are observed, inverse optimization has been developed and utilized to infer the underlying decision making problem. The majority of existing studies assumes that the decision making problem is with a single objective function, and attributes data divergence to noises, errors or bounded rationality, which, however, could lead to a corrupted inference when decisions are tradeoffs among multiple criteria. In this paper, we take a data-driven approach and design a more sophisticated inverse optimization formulation to explicitly infer parameters of a multiobjective decision making problem from noisy observations. This framework, together with our mathematical analyses and advanced algorithm developments, demonstrates a strong capacity in estimating critical parameters, decoupling "interpretable" components from noises or errors, deriving the denoised \emph{optimal} decisions, and ensuring statistical significance. In particular, for the whole decision maker population, if suitable conditions hold, we will be able to understand the overall diversity and the distribution of their preferences over multiple criteria, which is important when a precise inference on every single decision maker is practically unnecessary or infeasible. Numerical results on a large number of experiments are reported to confirm the effectiveness of our unique inverse optimization model and the computational efficacy of the developed algorithms.

For many years, a combination of principal component analysis (PCA) and independent component analysis (ICA) has been used as a blind source separation (BSS) technique to separate hidden sources of natural data. However, it is unclear why these linear methods work well because most real-world data involve nonlinear mixtures of sources. We show that a cascade of PCA and ICA can solve this nonlinear BSS problem accurately as the variety of input signals increases. Specifically, we present two theorems that guarantee asymptotically zero-error BSS when sources are mixed by a feedforward network with two processing layers. Our first theorem analytically quantifies the performance of an optimal linear encoder that reconstructs independent sources. Zero-error is asymptotically reached when the number of sources is large and the numbers of inputs and nonlinear bases are large relative to the number of sources. The next question involves finding an optimal linear encoder without observing the underlying sources. Our second theorem guarantees that PCA can reliably extract all the subspace represented by the optimal linear encoder, so that a subsequent application of ICA can separate all sources. Thereby, for almost all nonlinear generative processes with sufficient variety, the cascade of PCA and ICA performs asymptotically zero-error BSS in an unsupervised manner. We analytically and numerically validate the theorems. These results highlight the utility of linear BSS techniques for accurately recovering nonlinearly mixed sources when observations are sufficiently diverse. We also discuss a possible biological BSS implementation.