Fast Transformers with Clustered Attention Supplementary Material

Figure 1: Flow-chart demonstrating the compuation for clustered attention. For more details refer to 1.1 or 3.2 in the main paper. Work done at Idiap 34th Conference on Neural Information Processing Systems (NeurIPS 2020), V ancouver, Canada. We then present the flow chart demonstrating the same. This is followed by taking the weighted average of the 3 correponding values.