Self-supervised large language models have demonstrated the ability to perform various tasks via in-context learning, but little is known about where the model locates the task with respect to prompt instructions and demonstration examples. In this work, we attempt to characterize the region where large language models transition from recognizing the task to performing the task.

Inthesenetworks,attentionalignsencoderand decoder states and isoften used for visualizing network behavior.