A parametric activation function based on Wendland RBF

Activation functions play a pivotal role in neural networks by introducing non-linearity, allowing them to model complex data patterns. Common activation functions such as ReLU (Rectified Linear Unit), sigmoid, and tanh have been extensively used in various architectures. These functions provide effective solutions, yet they are not without limitations. For instance, ReLU is prone to dying ReLU problems, while sigmoid and tanh can suffer from vanishing gradients during training. As neural networks continue to grow in depth and complexity, the quest for more robust and efficient activation functions remains a critical area of research. In this paper, we propose a novel approach by introducing Wendland Radial Basis Functions (RBFs) as potential activation functions for neural networks. Wend-land RBFs, which are a class of smooth, compactly supported functions, offer a number of intriguing properties, such as locality and smoothness, which are crucial for improving model generalization and training efficiency. These functions have been successfully applied in interpolation and approximation tasks due to their mathematical stability and positive definiteness. We hypothesize that these properties can enhance neural networks, offering an alternative to traditional activation functions.