Technology
Parameters as interacting particles: long time convergence and asymptotic error scaling of neural networks
Grant Rotskoff, Eric Vanden-Eijnden
Theperformance ofneural networksonhigh-dimensional datadistributions suggests that it may be possible to parameterize a representation of agiven highdimensional function with controllably small errors, potentially outperforming standard interpolation methods. We demonstrate, both theoretically and numerically, that this is indeed the case. We map the parameters of a neural network to a system of particles relaxing with an interaction potential determined by the lossfunction.
Cost(Q,G,F) = MX
Todetermine the best threshold, we first determine the value ofT that results inthe best AI-alone accuracy on a small subset of the ImageNet-ReaL (2K images) and CUB (1K images) datasets, and then we evaluate the AI-alone accuracy on the held-out set for each dataset (42K images on ImageNet-ReaLand4KonCUB).