The advent of large language models (LLMs) [Brown et al., 2020] has revolutionized the field of

The ocean is a crucial component of the Earth's system.

Despite several works trying to reduce their computational cost, most of LLMs still adopt attention layers between all pairs of tokens in the sequence, thus incurring a quadratic cost. In this study, we present a novel approach that dynamically prunes contextual information while preserving the model's expressiveness, resulting in reduced memory and computational