Large Language Model
Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers Siyu Chen Department of Statistics and Data Science, Yale University
In particular, most existing work only theoretically explains how the attention mechanism facilitates ICL under certain data models.
Chain-of-Thought Reasoning without Prompting
These methods, while effective, often involve manually intensive prompt engineering.
OneBit: Towards Extremely Low-bit Large Language Models
LLMs difficult beyond mid-to-high-end GPUs like the A100, let alone on mobile devices. The high demand for resources not only drives up usage costs, but also restricts their wider application.
Terra: A Multimodal Spatio-Temporal Dataset Spanning the Earth Wei Chen
Since the inception of our planet, the meteorological environment, as reflected through spatio-temporal data, has always been a fundamental factor influencing human life, socio-economic progress, and ecological conservation.
cPAPERS: A Dataset of Situated and Multimodal Interactive Conversations in Scientific Papers
Besides textual paragraphs, researchers rely on various modalities to describe research methods. Figures convey information about concepts developed throughout the paper.
Amortized Planning with Large-Scale Transformers: A Case Study on Chess
This paper uses chess, a landmark planning problem in AI, to assess transformers'
