Overview
Block Transformer: Global-to-Local Language Modeling for Fast Inference
We introduce the Block Transformer which adopts hierarchical global-to-local modeling to autoregressive transformers to mitigate the inference bottlenecks associated with self-attention. Self-attention requires the key-value (KV) cache of all previous sequences to be retrieved from memory at every decoding step to retrieve context information, leading to two primary bottlenecks during batch inference. First, there is a significant delay in obtaining the first token, as the information of the entire prompt must first be processed to prefill the KV cache.
Continuous-time Value Function Approximation in Reproducing Kernel Hilbert Spaces
Motoya Ohnishi, Masahiro Yukawa, Mikael Johansson, Masashi Sugiyama
Neural Information Processing Systems http://nips.cc/
Data Parameters: A New Family of Parameters for Learning a Differentiable Curriculum
Shreyas Saxena, Oncel Tuzel, Dennis DeCoste
Neural Information Processing Systems http://nips.cc/
Easy2Hard-Bench: Standardized Difficulty Labels for Profiling LLM Performance and Generalization
Aiming to address this limitation, we present Easy2Hard-Bench, a consistently formatted collection of 6 benchmark datasets spanning various domains, such as mathematics and programming problems, chess puzzles, and reasoning questions.