Kolmogorov-Arnold Convolutions: Design Principles and Empirical Studies

The rapid evolution of deep learning architectures has significantly advanced the field of computer vision, particularly in tasks that require the analysis of complex spatial data. Convolutional Neural Networks (CNNs), initially proposed by LeCun et al. [1], have become a cornerstone in this domain due to their ability to efficiently process highdimensional data arrays such as images. These networks typically employ linear transformations followed by activation functions in their convolutional layers to discern spatial relationships, thereby reducing the number of parameters needed to capture intricate patterns in visual data. Since 2012, following the success of AlexNet [2] in the ImageNet classification challenge, CNNs have dominated the field of computer vision until the emergence of Vision Transformers [3]. Innovations such as Residual Networks [4] and Densely Connected networks [5], along with numerous subsequent works, have significantly advanced the achievable quality of models based on convolutional layers, enabling the effective training of very large and deep networks. In segmentation tasks, especially within the biomedical domain, CNNs have also become foundational with the advent of the U-Net [6] architecture, which has subsequently inspired a whole family of U-Net-like architectures for segmentation tasks. Recent developments in deep learning have seen the integration of sophisticated mathematical theories into neural network architectures, enhancing their capability to handle complex data structures. One such innovation is the Kolmogorov-Arnold Network (KAN) [7], which leverages the Kolmogorov-Arnold theorem to incorporate splines into its architecture, offering a compelling alternative to traditional Multi-Layer Perceptrons (MLPs).