r/MachineLearning - [P] Nearing BERT's accuracy on Sentiment Analysis with a model 56 times smaller by Knowledge Distillation

Should being comparable to BERT really be your goal here? The thing about BERT is that it wasn't really specifically designed for sentiment analysis. It just happens that it does that well too. But there's no reason to believe it's anywhere close to the "best way" to do sentiment analysis. I mean, as an analogy, pulling out a calculator to make a quick computation is often more convenient than booting up Matlab to do it, but using this fact to extol the merits of calculator kind of misses the point. If you want to describe how good your model is, you really should choose more relevant comparisons.