Top Data Scientist Claudia Perlich's Favorite Machine Learning Algorithm

I know that in the day and age of Deep Learning this seems to be a really odd answer. So let's start with a bit of background: In 1995–1998 I was using neural networks, 1998–2002 I was working mostly with tree based methods and from 2002 on, logistic regression (and linear models in general including quantile regression, Poisson regression, etc.) ended up to slowly make its way into my heart. In 2003 I published a paper in Machine Learning showing the results on comparing tree based methods against logistic regression on 35 (at the time large) datasets. The short answer (if you want to skip the 30 pages) - if the signal to noise ratio is high, trees tend to win. But, if you have very noisy problems and the best model has an AUC 0.8 - logistic beats the trees almost always.