r/MachineLearning - [R] Neural Oblivious Decision Ensembles

TL;DR: authors propose a DenseNet-like ensemble of decision trees, trained end-to-end by backpropagation and beats both xgboost and neural networks on heterogeneous ("tabular") data. Nowadays, deep neural networks (DNNs) have become the main instrument for machine learning tasks within a wide range of domains, including vision, NLP, and speech. Meanwhile, in an important case of heterogenous tabular data, the advantage of DNNs over shallow counterparts remains questionable. In particular, there is no sufficient evidence that deep learning machinery allows constructing methods that outperform gradient boosting decision trees (GBDT), which are often the top choice for tabular problems. In this paper, we introduce Neural Oblivious Decision Ensembles (NODE), a new deep learning architecture, designed to work with any tabular data.