Surfing: Iterative Optimization Over Incrementally Trained Deep Networks

We investigate a sequential optimization procedure to minimize the empiricalrisk functional fbθ(x) = 12kGbθ(x) yk2 for certain families of deep networks Gθ(x).