Ben Recht starts a blog • /r/MachineLearning

While I think nonconvex optimization is a very interesting area of research, I often feel like the community overstates its importance in ML. Finding a global optimum almost never happens in successful usage of neural nets (we use early-stopping with a validation set) and is not necessarily the best-idea for many applications. Rather, I feel that selecting a proper objective function, class of models, and regularization strategy are just as important considerations for ML. That said, much of the ML/statistical theory only holds when the empirical risk minimizer is actually found (M-estimation). One way to bridge this theoretical gap is of course to explicitly ensure you can find global optima via superior optimization or changing the model (eg.