Education
Unlocking Tokens as Data Points for Generalization Bounds on Larger Language Models
With Monarch matrices, Kronecker factorizations, and post-training quantization, we achieve non-vacuous generalization bounds for LLMs as large as LLaMA2-70B.
A Simple and Adaptive Learning Rate for FTRL in Online Learning with Minimax Regret of ฮ(T2 / 3) and its Application to Best-of-Both-Worlds
Follow-the-Regularized-Leader (FTRL) is a powerful framework for various online learning problems. By designing its regularizer and learning rate to be adaptive to past observations, FTRL is known to work adaptively to various properties of an underlying environment.
CiteME: Can Language Models Accurately Cite Scientific Claims?
Thousands of new scientific papers are published each month.
DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving Y uxuan Tong
In this work, we focus on mathematical problem-solving to explore enhancement of the mathematical reasoning abilities of pretrained LLMs. We investigate instruction tuning (Longpre et al., 2023; Wang
Understanding the Gains from Repeated Self-Distillation
Self-Distillation is a special type of knowledge distillation where the student model has the same architecture as the teacher model.
