Deep Learning
Kraken: Inherently Parallel Transformers For Efficient Multi-Device Inference
Parallelism introduces collective communication that is both expensive and represents a phase when hardware resources are underuti-lized.
CiteME: Can Language Models Accurately Cite Scientific Claims?
Thousands of new scientific papers are published each month.
DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving Y uxuan Tong
In this work, we focus on mathematical problem-solving to explore enhancement of the mathematical reasoning abilities of pretrained LLMs. We investigate instruction tuning (Longpre et al., 2023; Wang
Understanding the Gains from Repeated Self-Distillation
Self-Distillation is a special type of knowledge distillation where the student model has the same architecture as the teacher model.
MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models
Large Language Models (LLMs) are distinguished by their massive parameter counts, which typically result in significant redundancy.
