The Three Cultures of Machine Learning

Estimators in the previous two cases often have to solve intractable optimization problems, which leads to approximations and local maxima: you don't know quite what you'll get. But in simple settings, the errors of both approaches can be analyzed, which gratifies the people at the left vertex. Frequentist statisticians and COLT folks (computational learning theorists) cluster around that vertex; e.g., they solve convex optimization problems and try to bound the error. Examples include spectral learning, SVMs, and other convex or closed-form frequentist estimators.