Goto

Collaborating Authors

 Statistical Learning


Fast Gaussian Process Regression for Big Data

arXiv.org Machine Learning

Gaussian Processes are widely used for regression tasks. A known limitation in the application of Gaussian Processes to regression tasks is that the computation of the solution requires performing a matrix inversion. The solution also requires the storage of a large matrix in memory. These factors restrict the application of Gaussian Process regression to small and moderate size data sets. We present an algorithm that combines estimates from models developed using subsets of the data obtained in a manner similar to the bootstrap. The sample size is a critical parameter for this algorithm. Guidelines for reasonable choices of algorithm parameters, based on detailed experimental study, are provided. Various techniques have been proposed to scale Gaussian Processes to large scale regression tasks. The most appropriate choice depends on the problem context. The proposed method is most appropriate for problems where an additive model works well and the response depends on a small number of features. The minimax rate of convergence for such problems is attractive and we can build effective models with a small subset of the data. The Stochastic Variational Gaussian Process and the Sparse Gaussian Process are also appropriate choices for such problems. These methods pick a subset of data based on theoretical considerations. The proposed algorithm uses bagging and random sampling. Results from experiments conducted as part of this study indicate that the algorithm presented in this work can be as effective as these methods. Keywords: Big Data, Gaussian Process, Regression 2010 MSC: 00-01, 99-00 1. Introduction Gaussian Processes (GP) are attractive tools to perform supervised learning tasks on complex datasets on which traditional parametric methods may not be effective. They are also easier to use in comparison to alternatives like neural networks ([1]).


A probabilistic approach to emission-line galaxy classification

arXiv.org Machine Learning

We invoke a Gaussian mixture model (GMM) to jointly analyse two traditional emission-line classification schemes of galaxy ionization sources: the Baldwin-Phillips-Terlevich (BPT) and $\rm W_{H\alpha}$ vs. [NII]/H$\alpha$ (WHAN) diagrams, using spectroscopic data from the Sloan Digital Sky Survey Data Release 7 and SEAGal/STARLIGHT datasets. We apply a GMM to empirically define classes of galaxies in a three-dimensional space spanned by the $\log$ [OIII]/H$\beta$, $\log$ [NII]/H$\alpha$, and $\log$ EW(H${\alpha}$), optical parameters. The best-fit GMM based on several statistical criteria suggests a solution around four Gaussian components (GCs), which are capable to explain up to 97 per cent of the data variance. Using elements of information theory, we compare each GC to their respective astronomical counterpart. GC1 and GC4 are associated with star-forming galaxies, suggesting the need to define a new starburst subgroup. GC2 is associated with BPT's Active Galaxy Nuclei (AGN) class and WHAN's weak AGN class. GC3 is associated with BPT's composite class and WHAN's strong AGN class. Conversely, there is no statistical evidence -- based on four GCs -- for the existence of a Seyfert/LINER dichotomy in our sample. Notwithstanding, the inclusion of an additional GC5 unravels it. The GC5 appears associated to the LINER and Passive galaxies on the BPT and WHAN diagrams respectively. Subtleties aside, we demonstrate the potential of our methodology to recover/unravel different objects inside the wilderness of astronomical datasets, without lacking the ability to convey physically interpretable results. The probabilistic classifications from the GMM analysis are publicly available within the COINtoolbox (https://cointoolbox.github.io/GMM\_Catalogue/).


Identification of individual coherent sets associated with flow trajectories using Coherent Structure Coloring

arXiv.org Machine Learning

We present a method for identifying the coherent structures associated with individual Lagrangian flow trajectories even where only sparse particle trajectory data is available. The method, based on techniques in spectral graph theory, uses the Coherent Structure Coloring vector and associated eigenvectors to analyze the distance in higher-dimensional eigenspace between a selected reference trajectory and other tracer trajectories in the flow. By analyzing this distance metric in a hierarchical clustering, the coherent structure of which the reference particle is a member can be identified. This algorithm is proven successful in identifying coherent structures of varying complexities in canonical unsteady flows. Additionally, the method is able to assess the relative coherence of the associated structure in comparison to the surrounding flow. Although the method is demonstrated here in the context of fluid flow kinematics, the generality of the approach allows for its potential application to other unsupervised clustering problems in dynamical systems such as neuronal activity, gene expression, or social networks.


Pillar Networks++: Distributed non-parametric deep and wide networks

arXiv.org Machine Learning

In recent work, it was shown that combining multi-kernel based support vector machines (SVMs) can lead to near state-of-the-art performance on an action recognition dataset (HMDB-51 dataset). This was 0.4\% lower than frameworks that used hand-crafted features in addition to the deep convolutional feature extractors. In the present work, we show that combining distributed Gaussian Processes with multi-stream deep convolutional neural networks (CNN) alleviate the need to augment a neural network with hand-crafted features. In contrast to prior work, we treat each deep neural convolutional network as an expert wherein the individual predictions (and their respective uncertainties) are combined into a Product of Experts (PoE) framework.


Cost-Sensitive Reference Pair Encoding for Multi-Label Learning

arXiv.org Machine Learning

A general framework for multi-label classification(MLC) called multi-label error-correcting code (ML-ECC) utilizes coding schemes in communication to improve MLC performance. The framework includes some key algorithms for some special cases of MLC, such as binary relevance and random k-labelsets. Nevertheless, current ML-ECC algorithms are usually designed for one or a few evaluation criteria, and thus may suffer from bad performance with respect to other criteria. In this paper, we propose a ML-ECC algorithm that takes the evaluation criteria into account within the error-correcting code.This algorithm, named cost-sensitive reference pair encoding(CSRPE), first transforms the MLC problem into exponentially many binary classification problems based on the criterion information and a series of reduction steps from MLC to multi-class classification and then to binary classification. The exponentially many binary classifiers cause training and prediction challenges.We resolve the training challenge by random sampling and the prediction challenge by nearest-neighbor decoding. Extensive experimental results show that CSRPE achieves stable convergence, and performs better than other ML-ECC algorithms and the state-of-the-art cost-sensitive MLC algorithms across different criteria. Furthermore, we demonstrate the potential of CSRPE in preserving the criterion information by extending it to a novel multi-label active learning algorithm. The algorithm calculates the uncertainty of each unlabeled example in the coding space of CSRPE and queries the most uncertain one. Experimental results demonstrate that the proposed algorithm is superior to existing multi-label active learning algorithms.


Statistical Anomaly Detection via Composite Hypothesis Testing for Markov Models

arXiv.org Machine Learning

Under Markovian assumptions, we leverage a Central Limit Theorem (CLT) for the empirical measure in the test statistic of the composite hypothesis Hoeffding test so as to establish weak convergence results for the test statistic, and, thereby, derive a new estimator for the threshold needed by the test. We first show the advantages of our estimator over an existing estimator by conducting extensive numerical experiments. We find that our estimator controls better for false alarms while maintaining satisfactory detection probabilities. We then apply the Hoeffding test with our threshold estimator to detecting anomalies in two distinct applications domains: one in communication networks and the other in transportation networks. The former application seeks to enhance cyber security and the latter aims at building smarter transportation systems in cities.


Best Python books, courses, videos & tutorials 2017 - ReactDOM

#artificialintelligence

Python is a very popular high-level language created by Guido van Rossum and first released in 1991. Python is named after the greatest comedy act of all time, Monty Python. Python can be used to create pretty much any type of application. Python has been popular for many years and it's popularity shows no signs of stopping anytime soon. Been an in demand language, knowing Python is definitely something beneficial for your career as a software developer. Python is a very widely used programming languages that can do almost anything. Having working knowledge of high level programming languages is something any software developer should have. Whether it is a script you need to run or a complete application, Python is something you can use in your daily life as a programmer. Here's a list of some of the best Python books, courses, videos and tutorials in 2017 to help you learn Python.


The best kept secret about linear and logistic regression

@machinelearnbot

Regression can be performed as accurately without statistical models, including the computation of confidence intervals (for estimates, predicted values or regression parameters). It is indeed incredibly simple, robust, easy to interpret, and easy to code (no statistical libraries required). The exact theoretical solution and the one provided by traditional linear regression are almost as good, their predictive power is almost identical, as measured using the three metrics described in the previous section. But in each case, the estimated (predicted) values have similar correlation to the response: 0.69 for my approximation, and 0.72 for both theoretical and traditional solution.


How to Treat Missing Values in Your Data

@machinelearnbot

One of most excruciating pain points during Data Exploration and Preparation stage of an Analytics project are missing values. How do you deal with missing values - ignore or treat them? The answer would depend on the percentage of those missing values in the dataset, the variables affected by missing values, whether those missing values are a part of dependent or the independent variables, etc. Missing Value treatment becomes important since the data insights or the performance of your predictive model could be impacted if the missing values are not appropriately handled.The 2 tables above give different insights. The inference from the table on the left with the missing data indicates lower count for Android Mobile users and iOS Tablet users and higher Average Transaction Value compared to the inference from the right table with no missing data. The inference from the data with missing values could adversely impact business decisions.


Natural Language Processing: State of The Art, Current Trends and Challenges

arXiv.org Artificial Intelligence

Natural language processing (NLP) has recently gained much attention for representing and analysing human language computationally. It has spread its applications in various fields such as machine translation, email spam detection, information extraction, summarization, medical, and question answering etc. The paper distinguishes four phases by discussing different levels of NLP and components of Natural Language Generation (NLG) followed by presenting the history and evolution of NLP, state of the art presenting the various applications of NLP and current trends and challenges.