I know that in the day and age of Deep Learning this seems to be a really odd answer. So let's start with a bit of background: In 1995–1998 I was using neural networks, 1998–2002 I was working mostly with tree based methods and from 2002 on, logistic regression (and linear models in general including quantile regression, Poisson regression, etc.) ended up to slowly make its way into my heart. In 2003 I published a paper in Machine Learning showing the results on comparing tree based methods against logistic regression on 35 (at the time large) datasets. The short answer (if you want to skip the 30 pages) - if the signal to noise ratio is high, trees tend to win. But, if you have very noisy problems and the best model has an AUC 0.8 - logistic beats the trees almost always.

Having tried logistic regression the first time around, I moved on to decision trees and KNN. But unfortunately, those models performed horribly and had to be scrapped. Random Forest seemed to be the buzz word around the Kaggle forums, so I obviously had to try it out next. I took a couple of days to read up on it, worked out a few examples on my own before re-taking a stab at the titanic dataset. The'caret' package is a beauty.

In computer science, a logistic model tree (LMT) is a classification model with an associated supervised training algorithm that combines logistic regression (LR) and decision tree learning.[1][2] Logistic model trees are based on the earlier idea of a model tree: a decision tree that has linear regression models at its leaves to provide a piecewise linear regression model (where ordinary decision trees with constants at their leaves would produce a piecewise constant model).[1] In the logistic variant, the LogitBoost algorithm is used to produce an LR model at every node in the tree; the node is then split using the C4.5 criterion. Each LogitBoost invocation is warm-started[vague] from its results in the parent node. Finally, the tree is pruned.[3]

Our team won the second prize of the Safe Aging with SPHERE Challenge organized by SPHERE, in conjunction with ECML-PKDD and Driven Data. The goal of the competition was to recognize activities performed by humans, using sensor data. This paper presents our solution. It is based on a rich pre-processing and state of the art machine learning methods. From the raw train data, we generate a synthetic train set with the same statistical characteristics as the test set. We then perform feature engineering. The machine learning modeling part is based on stacking weak learners through a grid searched XGBoost algorithm. Finally, we use post-processing to smooth our predictions over time.

This research introduces a new strategy in cluster ensemble selection by using Independency and Diversity metrics. In recent years, Diversity and Quality, which are two metrics in evaluation procedure, have been used for selecting basic clustering results in the cluster ensemble selection. Although quality can improve the final results in cluster ensemble, it cannot control the procedures of generating basic results, which causes a gap in prediction of the generated basic results' accuracy. Instead of quality, this paper introduces Independency as a supplementary method to be used in conjunction with Diversity. Therefore, this paper uses a heuristic metric, which is based on the procedure of converting code to graph in Software Testing, in order to calculate the Independency of two basic clustering algorithms. Moreover, a new modeling language, which we called as "Clustering Algorithms Independency Language" (CAIL), is introduced in order to generate graphs which depict Independency of algorithms. Also, Uniformity, which is a new similarity metric, has been introduced for evaluating the diversity of basic results. As a credential, our experimental results on varied different standard data sets show that the proposed framework improves the accuracy of final results dramatically in comparison with other cluster ensemble methods.

This paper presents a semi-parametric algorithm for online learning of a robot inverse dynamics model. It combines the strength of the parametric and non-parametric modeling. The former exploits the rigid body dynamics equa- tion, while the latter exploits a suitable kernel function. We provide an extensive comparison with other methods from the literature using real data from the iCub humanoid robot. In doing so we also compare two different techniques, namely cross validation and marginal likelihood optimization, for estimating the hyperparameters of the kernel function.

Machine learning is kind of magic right? But is it the kind of magic that can make us rich? And I don't mean lucrative consulting gig rich, I mean digging valuable metals out of the ground rich. Also I'd been meaning to try out some transfer learning and looking around for a good topic to try it on. Transfer learning is where you take a pre-trained convolution (or other) network and use it for your task.

Have people not been doing this up till now? Seems fairly obvious to me. A similar technique I've used for fun is applying k-NN to each layer to see how well the classes are separated at this layer based on the Euclidean distance (or whatever other metric one wishes to employ). Note that technically k-NN doesn't need any training at all!

Professors Hastie and Tibshirani published "The Elements of Statistical learning: Data mining, inference and prediction", with Jerome Friedman (springer, 2001, second edition 2009). This book has received a terrific reception, with over 45,000 copies sold. Both presenters are actively involved in research in statistical learning methods, and are well-known not only in the statistics community but in the machine-learning, neural network and bioinformatics fields as well. Their newer book "An Introduction to Statistical Learning, with Applications in R" (with Gareth James and Daniela Witten, 2013) is also a best-seller, and has remained consistently in the top 10 in the Amazon categories "Mathematics and Statistics" and "Artificial Intelligence", with a five-star rating based on 84 customer reviews. Over the years they have become leaders in the statistical analysis of DNA microarrays, working with leading-edge biologists such as Patrick Brown of Stanford University, and David Botstein of Princeton.

Nonparametric estimation of a multivariate density estimation is tackled via a method which combines traditional local smoothing with a form of global smoothing but without imposing a rigid structure. Simulation work delivers encouraging indications on the effectiveness of the method. An application to density-based clustering illustrates a possible usage. Consider estimation of the probability density function f(·) of a continuous random variable in cases when a parametric formulation for f is not considered appropriate. Given a random sample drawn form f, a variety of nonparametric estimation methods are available.