Statistical Learning
Improving Generalization and Convergence by Enhancing Implicit Regularization Mingze Wang 1,3, Jinbo Wang 1, 3 Haotian He1,3 Zilin Wang 1
We show that IRE can be practically incorporated with generic base optimizers without introducing significant computational overload.
Sigmoid Gating is More Sample Efficient than Softmax Gating in Mixture of Experts
In particular, it aggregates multiple sub-models called experts based on a gating network. Here, experts can be formulated as neural networks, and they specialize in different aspects of the data.
Treeffuser: Probabilistic Predictions via Conditional Diffusions with Gradient-Boosted Trees Nicolas Beltran-V elez
Probabilistic prediction aims to compute predictive distributions rather than single point predictions. These distributions enable practitioners to quantify uncertainty, compute risk, and detect outliers. However, most probabilistic methods assume parametric responses, such as Gaussian or Poisson distributions.
Regression under demographic parity constraints via unlabeled post-processing
We address the problem of performing regression while ensuring demographic parity, even without access to sensitive attributes during inference. We present a general-purpose post-processing algorithm that, using accurate estimates of the regression function and a sensitive attribute predictor, generates predictions that meet the demographic parity constraint. Our method involves discretization and stochastic minimization of a smooth convex function.
Adaptive Sampling for Efficient Softmax Approximation
The softmax function is ubiquitous in machine learning and optimization applications.
