Research Highlights: Why Do Tree-based Models Still Outperform Deep Learning on Tabular Data? - insideBIGDATA

Title of paper: Why do tree-based models still outperform deep learning on tabular data? Abstract: While deep learning has enabled tremendous progress on text and image datasets, its superiority on tabular data is not clear. We contribute extensive benchmarks of standard and novel deep learning methods as well as tree-based models such as XGBoost and Random Forests, across a large number of datasets and hyperparameter combinations. We define a standard set of 45 datasets from varied domains with clear characteristics of tabular data and a benchmarking methodology accounting for both fitting models and finding good hyperparameters. Results show that tree-based models remain state-of-the-art on medium-sized data ( 10K samples) even without accounting for their superior speed.