Improved risk tail bounds for on-line algorithms

We prove the strongest known bound for the risk of hypotheses selected from the ensemble generated by running a learning algorithm incrementally onthe training data. Our result is based on proof techniques that are remarkably different from the standard risk analysis based on uniform convergence arguments.