TabNet: The End of Gradient Boosting?

Gradient Boosting models such as XGBoost, LightGBM and Catboost have long been considered best in class for tabular data. Even with rapid progress in NLP and Computer Vision, Neural Networks are still routinely surpassed by tree-based models on tabular data. Enter Google's TabNet in 2019. According to the paper, this Neural Network was able to outperform the leading tree based models across a variety of benchmarks. Not only that, it is considerably more explainable than boosted tree models as it has built-in explainability.