Large Margin Micro Blogging

While intricate deep models are all the rage in machine learning, my last post tried to make the case that we still need to care about model regularization. As it has been since the dawn of time, choosing simple models is still our best weapon against overfitting. But what does "simple" mean exactly? Of the many ways to measure simplicity, I've always been partial to the "minimum description length" principle: the fewer lines of code needed to train, store, and evaluate your model, the more robust that model likely is. Unfortunately, there aren't numerical methods that reliably compute models of minimum description length.