The Transformer architecture aggregates input information through the self-attention mechanism, but there is no clear understanding of how this information is mixed across the entire model. Additionally, recent works have demonstrated that attention weights alone are not enough to describe the flow of information. In this paper, we consider the whole attention block -- multi-head attention, residual connection, and layer normalization -- and define a metric to measure token-to-token interactions within each layer. Then, we aggregate layer-wise interpretations to provide input attribution scores for model predictions. Experimentally, we show that our method, ALTI (Aggregation of Layer-wise Token-to-token Interactions), provides more faithful explanations and increased robustness than gradient-based methods.

ALTI is a world-leading unmanned aircraft developer and manufacturing company offers the most advanced VTOL (vertical take-off and land) unmanned aircraft systems available. The company is the largest commercial drone manufacturer in Africa, based in South Africa, manufacturing and developing unmanned VTOL aircrafts since 2009 and exporting the most advanced products for the last 8 years. The ALTI Transition VTOL is the principal product of the company, but now introducing ALTI Reach the'bigger brother' to the Transition, aesthetically and configurationally very similar to the Transition, offering vertical take-off and landing and efficient fixed wing flight performance.