Transformers are uninterpretable with myopic methods: a case study with bounded Dyck grammars

Transformer interpretability aims to understand the algorithm implemented by a learned Transformer by examining various aspects of the model, such as the weight matrices or the attention patterns.