A A proof of the PAC Bayes Bennett inequality Theorem 9 and a comparison with the PAC Bayes Bernstein inequality

In this section we provide a proof of Theorem 9 and a numerical comparison with the P AC-Bayes-Bernstein inequality. The proof is based on the standard change of measure argument. The second ingredient is Bennett's lemma, which is a bound on the moment generating function used Now we are ready to prove the theorem. Therefore, for µ < 0 .5 we have null In this section we provide technical details on minimization of the bounds in Theorems 12 and 15. As most of the other P AC-Bayesian works, we take π to be a union distribution over the hypotheses 14 in both cases.