uGMM-NN: Univariate Gaussian Mixture Model Neural Network

Deep neural networks have transformed machine learning, excelling in tasks such as image classification and natural language processing through hierarchical feature learning [1]. However, traditional neurons, which compute deterministic weighted sums followed by nonlinear activations (e.g., ReLU, sigmoid), struggle to model uncertainty or multimodal distributions prevalent in real-world data. This limitation has historically been addressed by probabilistic graphical models, such as Bayesian Networks [2] and Markov Random Fields [3], which offer robust frameworks for uncertainty quantification and complex dependency modeling [4]. These models provide a strong conceptual foundation, but often lack the deep hierarchical feature learning capabilities of modern neural networks. A key research focus has therefore been on bridging the gap between these two paradigms. This has led to approaches that incorporate the probabilistic principles of graphical models directly into deep learning architectures. For example, Bayesian Neural Networks (BNNs) embed uncertainty into network weights [5], while Probabilistic Circuits (PCs), including Sum-Product Networks (SPNs) [7, 8, 9], are deep probabilistic models that build on a formal probabilistic structure, fusing the representational power of graphical models with the hierarchical feature learning of neural networks. In contrast, this paper introduces a novel approach by embedding a univariate Gaussian Mixture Model (uGMM) directly into the network's computational units, enabling each neuron to represent 1