model complexity
A Rigorous, Tractable Measure of Model Complexity
Allerbo, Oskar, Schön, Thomas B.
One of the most fundamental properties of a machine learning model is its complexity, with applications across topics such as interpretation, generalization, and model selection. Despite its importance, there is no canonical, model-agnostic way to assess a model's complexity. While simple heuristics, such as the number or magnitude of parameters, yield very crude estimates, hyperparameter-based approaches, such as polynomial degree or kernel length scale, do not generalize across model classes. More rigorous methods, including the Vapnik-Chervonenkis dimension (VCD) (Vapnik, 2013), Rademacher complexity (RMC) (Bartlett and Mendelson, 2002), and effective number of parameters (or effective degrees of freedom, ENP) (Efron, 1986), are difficult, or even impossible, to compute in practice, leaving the user to resort to crude bounds and/or approximations. The topic is further complicated by the often overlooked distinction between model and function complexity, where the former sets a ceiling on the latter.
Neural approximation of Wasserstein distance via a universal architecture for symmetric and factorwise group invariant functions
Learning distance functions between complex objects, such as the Wasserstein distance to compare point sets, is a common goal in machine learning applications. However, functions on such complex objects (e.g., point sets and graphs) are often required to be invariant to a wide variety of group actions e.g.
Reinforced Continual Learning
Neural Information Processing Systems http://nips.cc/
Optimal Statistical Rates for Decentralised Non-Parametric Regression with Linear Speed-Up
Dominic Richards, Patrick Rebeschini
Due to the increasingly large size of datasets and high model complexity, direct minimisation of the regularised problem is posing more and morecomputationalchallenges.
Transfer Learning via Minimizing the Performance Gap Between Domains
Boyu Wang, Jorge Mendez, Mingbo Cai, Eric Eaton
To address this issue, we present the first analysis for instance weighting transfer learning that considers the presence of labeled target examples. The contribution of our work is two-fold.1. We address the question ofhow to measure the divergence between two domains given label informationforthetargetdomain.
Multi-value Rule Sets for Interpretable Classification with Feature-Efficient Representations
Neural Information Processing Systems http://nips.cc/
Supervised autoencoders: Improving generalization performance with unsupervised regularizers
Lei Le, Andrew Patterson, Martha White
Neural Information Processing Systems http://nips.cc/
AMathematicalFrameworkforQuantifying TransferabilityinMulti-sourceTransferLearning
Therefore, forasource task withacomplexmodel orfewtraining samples, even though itis similar to the target task, the knowledge transferable from this source task can still be verylimited.
