Leveraging Intrinsic Gradient Information for Further Training of Differentiable Machine Learning Models

This work presents methods demonstrating that when the derivatives of target variables (outputs) with respect to inputs can be extracted - We introduce a novel metric that can be utilised in a from processes of interest, e.g., neural networks hyper-parameter optimisation pipeline that provides an (NN) based surrogate models, they can be leveraged indicator of an upper bound to NN model complexity to further improve the accuracy of differentiable - We propose an alternative regularisation method for linear ML models. This paper generalises the idea regression problems (using ridge regression as an and provides practical methodologies that can be example) that outperforms conventional regularisation used to leverage gradient information (GI) across over varying training sample sizes by utilising GI a variety of applications including: (1) Improving the performance of generative adversarial networks In the rest of this paper, Section 2 formulates the GI idea (GANs); (2) efficiently tuning NN model under a supervised learning setting. The proposed GI assisted complexity; (3) regularising linear regressions. Numerical methodologies are presented between Section 3 to 5, and followed results show that GI can effective enhance by a conclusion in Section 6. ML models with existing datasets, demonstrating its value for a variety of applications.