Deep neural networks have been successfully trained with simple gradient-based methods, despite the inherent non-convexity of the objectivefunction.

To handle the issue, we propose a new framework, coined robust model adaptation (RoMA), based on gradient-based optimization of inputs over the DNN. Specifically, it consists oftwosteps: (a)apre-training strategytorobustly train theproxy model and (b) a novel adaptation procedure of the proxy model to have robust estimates for a specific set of candidate solutions. At ahigh level, our scheme utilizes thelocal smoothness priorto overcome the brittleness of the DNN.