Assessing Simplification Levels in Neural Networks: The Impact of Hyperparameter Configurations on Complexity and Sensitivity

Motivation: The advantage of Lempel-Ziv complexity is that it can be approximated using existing compression algorithms This report aims to assess the simplification level of neural (such as zlib) and effectively represents the structure networks under different hyperparameter configurations, focusing and redundancy of the data, making it a suitable tool for particularly on two key metrics: Lempel-Ziv complexity assessing neural network output complexity. By adjusting activation functions, the number other complexity calculation methods, Lempel-Ziv complexity of hidden layers, and the learning rate of the neural network, is more efficient for evaluating network outputs and is straightforward we analyze how these hyperparameters impact the network's to compute. The experiment uses the MNIST dataset for a classification task, B. Sensitivity evaluating how networks configured with various hyperparameters Sensitivity measures the network's response to input perturbations. The number Motivation: We chose the L2 norm to calculate the magnitude of hidden layers is directly related to the model's capacity, of output perturbations because the L2 norm intuitively while the learning rate affects the model's convergence speed measures the Euclidean distance between two vectors, making and training quality [2]. By adjusting these parameters, we aim it well-suited for evaluating changes in neural network output.