In 1945, Count Richard Taaffe*, a Dublin gem collector, was sorting through a set of spinel gems that he had bought, and found one that refracted light differently - instead of simply bending light rays, it split them into two rays ("double refraction"). The anomalous gem was named after him and earned a place on the "world's rarest gems" list. In analytics, it sometimes not the rule (i.e. the model) that is of interest, but rather the exception. Detecting anomalous cases in large datasets is critical in conducting surveillance, countering credit-card fraud, protecting against network hacking, combating insurance fraud, and many more applications in government, business and healthcare. The techniques of anomaly detection are not new to the era of Big Data. Dr. Nitin Indurkhya, who teaches the Anomaly Detection course at Statistics.com, told me of an interesting application of anomaly detection to data that long pre-dates the era of Big Data.

Following my post about logistic regressions, Ryan got in touch about one bit of building logistic regressions models that I didn't cover in much detail – interpreting regression coefficients. This post will hopefully help Ryan (and others) out. I'd love to see more about interpreting the glm coefficients. Coefficients are what a line of best fit model produces. A line of best fit (aka regression) model usually consist of an intercept (where the line starts) and the gradients (or slope) for the line for one or more variables.

The link between different psychophysiological measures during emotion episodes is not well understood. To analyse the functional relationship between electroencephalography (EEG) and facial electromyography (EMG), we apply historical function-on-function regression models to EEG and EMG data that were simultaneously recorded from 24 participants while they were playing a computerised gambling task. Given the complexity of the data structure for this application, we extend simple functional historical models to models including random historical effects, factor-specific historical effects, and factor-specific random historical effects. Estimation is conducted by a component-wise gradient boosting algorithm, which scales well to large data sets and complex models.

Over the last few years, there was a surge of interest in the development of processing tools for the analysis of signals defined over a graph, or graph signals for short, in view of the many potential applications spanning from sensor networks, social media, vehicular networks, big data or biological networks [1]-[3]. Graph signal processing (GSP) considers signals defined over a discrete domain having a very general structure, represented by a graph, and subsumes classical discrete-time signal processing as a very simple case. Several processing methods for signals defined over a graph were proposed in [2], [4]-[6], and one of the most interesting aspects is that these analysis tools come to depend on the graph topology. A fundamental role in GSP is of course played by spectral analysis, which passes through the definition of the Graph Fourier Transform (GFT). Two main approaches for GFT have been proposed in the literature, based on the projection of the signal onto the eigenvectors of either the graph Laplacian, see, e.g., [1], [7], [8], or of the adjacency matrix, see, e.g.

Thanks to great experimental work by several research groups studying the behavior of Stochastic Gradient Descent (SGD), we are collectively gaining a much clearer understanding as to what happens in the neighborhood of training convergence. The story begins with the best paper award winner for ICLR 2017, "Rethinking Generalization". This paper I first discussed several months ago in a blog post "Rethinking Generalization in Deep Learning". One interesting observation in that paper is the role of SGD. Indeed, in neural networks, we almost always choose our model as the output of running stochastic gradient descent.

No need to apologize for not using "proper" weights.

Hello, I am working on a highly imbalanced dataset (negative examples over 20K and positive examples about 100). I am trying to build a logistic regression model. My current approach includes undersampling of negative examples. However with this approach there are a couple of problems: 1) Several LR models are possible with different samples. How to generalize these models and interpret the output?

Support vector machines (SVMs) are an important tool in modern data analysis. Traditionally, support vector machines have been fitted via quadratic programming, either using purpose-built or off-the-shelf algorithms. We present an alternative approach to SVM fitting via the majorization--minimization (MM) paradigm. Algorithms that are derived via MM algorithm constructions can be shown to monotonically decrease their objectives at each iteration, as well as be globally convergent to stationary points. We demonstrate the construction of iteratively-reweighted least-squares (IRLS) algorithms, via the MM paradigm, for SVM risk minimization problems involving the hinge, least-square, squared-hinge, and logistic losses, and 1-norm, 2-norm, and elastic net penalizations. Successful implementations of our algorithms are presented via some numerical examples.

The two main types of machine learning algorithms are supervised and unsupervised learning. There are many types of supervised algorithms available, one of the most popular ones is the Naive Bayes model which is often a good starting point for developers since it's fairly easy to understand the underlying probabilistic model and easy to execute. Decision trees are also a predictive model and have two types of trees: regression (which take continuous values) and classification models (which take finite values) and use a divide and conquer strategy that recursively separates the data to generate the tree. Check out the rest of the blog for more resources on natural language processing and machine learning algorithms such as LDA for text classification or increasing the accuracy on a Nudity Detection algorithm and a beginners tutorial on using Scikit-learn to solve FizzBuzz.

In last week's post I explored whether machine learning models can be applied to predict flu deaths from the 2013 outbreak of influenza A H7N9 in China. There, I compared random forests, elastic-net regularized generalized linear models, k-nearest neighbors, penalized discriminant analysis, stabilized linear discriminant analysis, nearest shrunken centroids, single C5.0 tree and partial least squares. Extreme gradient boosting (XGBoost) is a faster and improved implementation of gradient boosting for supervised learning and has recently been very successfully applied in Kaggle competitions. Because I've heard XGBoost's praise being sung everywhere lately, I wanted to get my feet wet with it too. So this week I want to compare the prediction success of gradient boosting with the same dataset.