Statistical Learning
Hunting for Discriminatory Proxies in Linear Regression Models
Samuel Yeom, Anupam Datta, Matt Fredrikson
Neural Information Processing Systems http://nips.cc/
Meta-Learning MCMC Proposals
Tongzhou Wang, YI WU, Dave Moore, Stuart J. Russell
Neural Information Processing Systems http://nips.cc/
Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data
Neural networks have many successful applications, while much less theoretical understanding has been gained. Towards bridging this gap, we study the problem of learning a two-layer overparameterized ReLU neural network for multi-class classification via stochastic gradient descent (SGD) from random initialization. In the overparameterized setting, when the data comes from mixtures of well-separated distributions, we prove that SGD learns a network with a small generalization error, albeit the network has enough capacity to fit arbitrary labels. Furthermore, the analysis provides interesting insights into several aspects of learning neural networks and can be verified based on empirical studies on synthetic data and on the MNIST dataset.
Multi-Layered Gradient Boosting Decision Trees
Ji Feng, Yang Yu, Zhi-Hua Zhou
Neural Information Processing Systems http://nips.cc/
CatBoost: unbiased boosting with categorical features
Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, Andrey Gulin
We show in this paper that all existing implementations of gradient boosting face the following statistical issue.
Learning SMaLL Predictors
Vikas Garg, Ofer Dekel, Lin Xiao
We introduce a new framework for learning in severely resource-constrained settings. Our technique delicately amalgamates the representational richness of multiple linear predictors with the sparsity of Boolean relaxations, and thereby yields classifiers that are compact, interpretable, and accurate. We provide a rigorous formalism of the learning problem, and establish fast convergence of the ensuing algorithm via relaxation to a minimax saddle point objective.
Appendix
A.4 EstimatingparameterswhenY(t)isunavailable New parameter estimators that leverage only the available data need to be derived whenY(t) is unavailable. The derivation goes as follows: first, we eliminateY(t) from the model equations. The squared error of the estimated parameters are shown in Figure 1. First, we estimated the parameters separately for each individual. Second, we performed statistical analysis to find associations between the estimated parameters and the demographic variables.
