[D] Benchmarks and algorithms for small-data regime? • r/MachineLearning

I'm working on a transfer learning thing that is geared towards optimizing the space of models to look at given that you know something about e.g. the amount of training data you will receive, examples of the kind of covariate shift or nonstationarity effects you expect to exist, etc. It's a kind of learned regularization trick, and the best results seem to be when you know you'll only ever have from 10-100 points of training data (you can also run it in unsupervised or semi-supervised modes, asking it to do the best it can with 10 labeled points and 100 points with no labels for example). The aim is to submit to NIPS 2018. Currently when I do performance comparisons I've mostly used the basic Kaggle standbys - linear SVC, RBF-kernel SVC, kNN, RandomForest, and XGBoost. What kinds of algorithms would you (as a reviewer) expect to see comparisons with in this problem space?