Entropy-based Guidance of Deep Neural Networks for Accelerated Convergence and Improved Performance

In the past 15 years, neural networks have revolutionized our capabilities in computer vision using convolutional neural networks (CNNs), for example, seminal works by LeCun et al. [28] and Krizhevsky et al. [27], and vision Transformers from Dosovitskiy et al. [10], natural language processing (e.g., by Radford et al. [38], Ouyang et al. [35]) with Transformers from Vaswani et al. [45], synthetic data generation with generative adversarial networks (GANs) from Goodfellow et al. [15] and diffusion models introduced by Ho et al. [23], and innumerable other domains. Neural networks have become essential tools in numerous fields, ranging from healthcare to national security, due to their exceptional predictive power and versatility. However, the opaque nature of these models poses significant challenges, particularly in high-stakes environments where understanding and trust in how decisions are made are crucial. Developing more interpretable neural network models could revolutionize their use, providing clear, actionable insights into why and how certain decisions are made. This clarity could enhance the efficiency of the models, improve collaboration with domain experts, and deepen our understanding of critical features and their impacts.