Goto

Collaborating Authors

 Deep Learning







Representational Strengths and Limitations of Transformers

Neural Information Processing Systems

In recent years, transformer networks [V aswani et al., 2017] have been established as a fundamental neural architecture powering state-of-the-art results in many applications, including language


Representational Strengths and Limitations of Transformers

Neural Information Processing Systems

In recent years, transformer networks [V aswani et al., 2017] have been established as a fundamental neural architecture powering state-of-the-art results in many applications, including language




The Learnability of In-Context Learning

Neural Information Processing Systems

Our theoretical analysis reveals that in this setting, in-context learning is more about identifying the task than about learning it, a result which is in line with a series of recent empirical findings.