Education
Understanding and Minimising Outlier Features in Transformer Training Bobby He
Despite their widespread use, our understanding of deep neural networks (NNs) and their training dynamics is very much incomplete.
Scaling Law for Time Series Forecasting
Scaling law that rewards large datasets, complex models and enhanced data granularity has been observed in various fields of deep learning.
GACL: Exemplar-Free Generalized Analytic Continual Learning
Class incremental learning (CIL) trains a network on sequential tasks with separated categories in each task but suffers from catastrophic forgetting, where models quickly lose previously learned knowledge when acquiring new tasks.
REDUCR: Robust Data Downsampling using Class Priority Reweighting
The abundance of data has had a profound impact on machine learning (ML), both positive and negative.
Reawakening knowledge: Anticipatory recovery from catastrophic interference via structured training Y anlai Y ang 1, Matt Jones
The behavior emerges and becomes more robust as the architecture scales up its number of parameters.
Bayes-optimal learning of an extensive-width neural network from quadratically many samples
Technically, our result is enabled by establishing a link with recent works on optimal denoising of extensive-rank matrices and on the ellipsoid fitting problem.
Monoculture in Matching Markets
This emerging literature has been limited by the challenge of incorporating market effects.