Appendix Table of Contents

The number of layers is 12 for GPT2 and randomly initialized model and 24 for iGPT. Note that these notations are sometimes used interchangeably as long as it doesn't significantly The activation to be analyzed are outputs from all layers . CKA about is shown in Figure 1. The design of the diagram is based on a previous study [35]. Figure 11: Activation we consider to compute CKA.