Asia
Enhancing In-Context Learning Performance with just SVD-Based Weight Pruning: A Theoretical Perspective
Pre-trained large language models (LLMs) based on Transformer have demonstrated striking in-context learning (ICL) abilities.
A Kernel Loss for Solving the Bellman Equation
Yihao Feng, Lihong Li, Qiang Liu
Neural Information Processing Systems http://nips.cc/
ImprovedFine-TuningbyBetterLeveraging Pre-TrainingData
As a dominant paradigm, fine-tuning a pre-trained model on the target data is widely used in many deep learning applications, especially for small data sets.
vGraph: A Generative Model for Joint Community Detection and Node Representation Learning
Fan-Yun Sun, Meng Qu, Jordan Hoffmann, Chin-Wei Huang, Jian Tang
Neural Information Processing Systems http://nips.cc/
ErrorCompensatedDistributedSGD canbeAccelerated
In this work, we show for the first time that error compensated gradient compression methods can be accelerated. In particular, we propose and study the error compensated loopless Katyusha method, and establish an accelerated linear convergence rate under standard assumptions.