Latent Space Topology Evolution in Multilayer Perceptrons

The widespread deployment of neural networks in critical decision-making systems has created an urgent need for interpretable machine learning models. While these architectures demonstrate remarkable empirical success across diverse domains, their internal mechanisms remain largely opaque, earning them the notorious designation as "black boxes". This opacity originates from the confluence of several fundamental challenges: the high-dimensional nature of parameter spaces, the compositional complexity introduced by multiple layers of non-linear transformations, and the emergent behaviours that arise from the interplay between architecture and optimisation dynamics. In this work, we focus on Multilayer Perceptrons (MLPs), the foundational architecture underlying modern deep learning. Despite their apparent simplicity compared to contemporary architectures, MLPs remain ubiquitous as essential components in more complex models. They appear as dense layers in Convolutional Neural Networks (CNNs), as projection heads in Vision Transformers, and as feed-forward networks in Transformer blocks. Understanding the internal representations learned by MLPs thus provides a gateway to interpreting broader classes of neural architectures. Moreover, in safety-critical applications such as medical diagnosis, financial risk assessment, and autonomous systems, the ability to interpret MLP decisions is highly important. The challenge of neural network interpretability has two complementary research directions.