Recent advances in attention-free sequence models rely on convolutions as alternatives to the attention operator at the core of Transformers. In particular, long convolution sequence models have achieved state-of-the-art performance in many domains, but incur a significant cost during auto-regressive inference workloads - naively requiring a full pass (or caching of activations) over the input sequence for each generated token - similarly to attention-based models. In this paper, we seek to enable O (1) compute and memory cost per token in any pre-trained long convolution architecture to reduce memory footprint and increase throughput during generation. Concretely, our methods consist in extracting low-dimensional linear state-space models from each convolution layer, building upon rational interpolation and model-order reduction techniques. We further introduce architectural improvements to convolution-based layers such as Hyena: by weight-tying the filters across channels into heads, we achieve higher pre-training quality and reduce the number of filters to be distilled. The resulting model achieves 10 higher throughput than Transformers and 1 .5 higher than Hyena at 1 .3

Specifically, we first propose a unified criteria to evaluate and certify the robustness of GCL.

Multi-temporal resolution: Sentinel-2 provides a consistent yearly time series. Sentinel-2 sensor acquires images at 10, 20 and 60 m spatial resolutions. T able 2: Provided cloud and snow masks. Note that cloudy dates are not suppressed from the time series. The spatial size of Sentinel-2 time series has been empirically determined and set to 40.

Information (IGN) that provides a unique and rich resource for large-scale geospa-tial analysis.

We adopt the natural assumption that the empirical NTK develops a block structure aligned with the class labels, i.e., samples within the same class have stronger

These authors contributed equally to this work.

Despite empirical advances, the theoretical study of frequentist LFI is still in its infancy.