The Unreasonable Effectiveness of Fully-Connected Layers for Low-Data Regimes

We perform classification experiments for a large range of network backbones and several standard datasets on supervised learning and active learning.