Choosing the most adequate kernel is crucial in many Machine Learning applications. Gaussian Process is a state-of-the-art technique for regression and classification that heavily relies on a kernel function. However, in the Gaussian Process literature, kernels have usually been either ad hoc designed, selected from a predefined set, or searched for in a space of compositions of kernels which have been defined a priori. In this paper, we propose a Genetic-Programming algorithm that represents a kernel function as a tree of elementary mathematical expressions. By means of this representation, a wider set of kernels can be modeled, where potentially better solutions can be found, although new challenges also arise. The proposed algorithm is able to overcome these difficulties and find kernels that accurately model the characteristics of the data. This method has been tested in several real-world time-series extrapolation problems, improving the state-of-the-art results while reducing the complexity of the kernels.

Causal inference is a fundemental problem in statistics and has wide applications in different fields. Transfer Entropy (TE) is a important notion defined for measuring causality, which is essentially conditional Mutual Information (MI). Copula Entropy (CE) is a theory on measurement of statistical independence and is equivalent to MI. In this paper, we prove that TE can be represented with only CE and then propose a non-parametric method for estimating TE via CE. The proposed method was applied to analyze the Beijing PM2.5 data in the experiments. Experimental results show that the proposed method can infer causality relationships from data effectively and hence help to understand the data better.

Deep neural networks are vulnerable to adversarial attacks and hard to interpret because of their black-box nature. The recently proposed invertible network is able to accurately reconstruct the inputs to a layer from its outputs, thus has the potential to unravel the black-box model. An invertible network classifier can be viewed as a two-stage model: (1) invertible transformation from input space to the feature space; (2) a linear classifier in the feature space. We can determine the decision boundary of a linear classifier in the feature space; since the transform is invertible, we can invert the decision boundary from the feature space to the input space. Furthermore, we propose to determine the projection of a data point onto the decision boundary, and define explanation as the difference between data and its projection. Finally, we propose to locally approximate a neural network with its first-order Taylor expansion, and define feature importance using a local linear model. We provide the implementation of our method: \url{https://github.com/juntang-zhuang/explain_invertible}.

Distributed representations have been used to support downstream tasks in healthcare recently. Healthcare data (e.g., electronic health records) contain multiple modalities of data from heterogeneous sources that can provide complementary information, alongside an added dimension to learning personalized patient representations. To this end, in this paper we propose a novel unsupervised encoder-decoder model, namely Mixed Pooling Multi-View Attention Autoencoder (MPVAA), that generates patient representations encapsulating a holistic view of their medical profile. Specifically, by first learning personalized graph embeddings pertaining to each patient's heterogeneous healthcare data, it then integrates the non-linear relationships among them into a unified representation through multi-view attention mechanism. Additionally, a mixed pooling strategy is incorporated in the encoding step to learn diverse information specific to each data modality. Experiments conducted for multiple tasks demonstrate the effectiveness of the proposed model over the state-of-the-art representation learning methods in healthcare.

Automatic text summarization (ATS) has recently achieved impressive performance thanks to recent advances in deep learning and the availability of large-scale corpora. To make the summarization results more faithful, this paper presents an unsupervised approach that combines rhetorical structure theory, deep neural model and domain knowledge concern for ATS. This architecture mainly contains three components: domain knowledge base construction based on representation learning, attentional encoder-decoder model for rhetorical parsing and subroutine-based model for text summarization. Domain knowledge can be effectively used for unsupervised rhetorical parsing thus rhetorical structure trees for each document can be derived. In the unsupervised rhetorical parsing module, the idea of translation was adopted to alleviate the problem of data scarcity. The subroutine-based summarization model purely depends on the derived rhetorical structure trees and can generate content-balanced results. To evaluate the summary results without golden standard, we proposed an unsupervised evaluation metric, whose hyper-parameters were tuned by supervised learning. Experimental results show that, on a large-scale Chinese dataset, our proposed approach can obtain comparable performances compared with existing methods.

If you love to explore large and challenging data sets, then probably you should give Microsoft Malware Classification a try. Before diving deep in to the problem let's take few points on what can you expect to learn from this: In the past few years, the malware industry has grown very rapidly that, the syndicates invest heavily in technologies to evade traditional protection, forcing the anti-malware groups/communities to build more robust software to detect and terminate these attacks. The major part of protecting a computer system from a malware attack is to identify whether a given piece of file/software is a malware. We can map the business problem to a multi-class classification problem, where we need to predict the class for each given byte files among nine categories (Ramnit, Lollipop, Kelihos_ver3, Vundo, Simda,Tracur, Kelihos_ver1, Obfuscator.ACY, Gatak). Constrains: We need to provide the class probability, wrongly classified class labels should be penalized(that's why log loss has been chosen as KPI) and there should some latency bound.

The algorithm starts off with setting initial values for coefficients -- you are free to set the values to whatever you like (just not a string or boolean), but the common practice is to set them to 0. If I have two coefficients, let's say beta 0 and beta 1, I would set them to zero initially: Now just to keep things simple let's say I'm dealing with a linear regression task, and those betas are my coefficients (beta 0 being the bias intercept). It's quite simple to read. You make a prediction, then subtract that prediction from the actual value, and you take the square of that. Now comes the part where you should know a bit of Calculus to fully understand what's going on. You need to calculate partial derivatives for each of the coefficients, so the coefficients can be updated later. Some time ago I've written an article on taking derivatives in Python, and it covers to a degree those topics: As my model has only two coefficients, I need to calculate two partial derivatives, one with respect to beta 0, and the other with respect to beta 1. Here's how: Now comes the part in which you take those two functions and do something known as epoch -- just a fancy word for a single iteration through the dataset.

This book explains the concept of machine learning starting from the very basics of Linear Regression and Logistic Regression, and ends at Multilevel Perceptrons to do Image Recognition. The best part about this book is that it assumes no prior knowledge in machine learning or even computer programming. The only basic requirement I see is the ability read basic English and the basic knowledge of high school level math. The author has also provided preprocessed data sets and a github repository, hence it is easy to start getting your hands dirty as soon as possible. This book is quite basic, but does the most crucial job of getting even the most layman to get excited about the field of Machine Learning and Deep Learning.

In this article, we will explore FisherFaces techniques of Face Recognition. FisherFaces is an improvement over EigenFaces and uses Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA). OpenCV has three built-in face recognizers. We can use any of them by a single line of code. In this article, we will focus on FisherFaces.

Last year at the Ignition Community conference, Inductive Automation's Kevin McClusky (co-director of sales engineering) and Kathy Applebaum (senior software engineer) explored the various ways in which machine learning can be applied in industry. In this presentation, they delved deep into the types of machine learning most applicable to industry and the algorithms behind them. You can read more about this 2018 presentation in the article "How to Apply Industrial Machine Learning," which was based on that presentation. At this year's event, McClusky and Applebaum came together again to highlight the integration of more machine learning capabilities into Ignition over the past year, as well as to showcase four industrial use cases of machine learning being explored by Ignition users. Newly available machine learning capabilities in Ignition enable users to take advantage of the Apache Math 3 library previously added to Ignition 7.9.10 just prior to the release of Ignition 8.