Optimizing Neural Networks with Learnable Non-Linear Activation Functions via Lookup-Based FPGA Acceleration

--Learned activation functions in models like Kolmogorov-Arnold Networks (KANs) outperform fixed-activation architectures in terms of accuracy and interpretability; however, their computational complexity poses critical challenges for energy-constrained edge AI deployments. Conventional CPUs/GPUs incur prohibitive latency and power costs when evaluating higher order activations, limiting deployability under ultra-tight energy budgets. We address this via a reconfigurable lookup architecture with edge FPGAs. FPGA reconfigurability enables dynamic hardware specialization for learned functions, a key advantage for edge systems that require post-deployment adaptability. This breakthrough positions our approach as a practical enabler for energy-critical edge AI, where computational intensity and power constraints traditionally preclude the use of adaptive activation networks. The development of effective activation functions has long been a central focus in machine learning research to enhance neural network capabilities. Neural networks with trainable activation functions represent an important and actively explored class of models, attracting growing research interest due to their potential to enhance model expressivity and adaptability to specific tasks [1] - complementing models with traditional fixed functions such as ReLU [2] and Leaky ReLU [3]. Learnable activation functions can be classified into two main categories: parameterized standard activation functions and ensemble-based activation functions [4].