48237d9f2dea8c74c2a72126cf63d933-AuthorFeedback.pdf

Ouranalysishelps13 illustrate why this worst-case behavior does not attain in practice. The standard union bound does not take advantage of model similarity; the refinement in equation (3) is the24 basis for our calculations that highlight the beneficial effect of model similarity. Our bounds demonstrate a link between similarity and protection against overfitting in both the30 non-adaptiveandadaptivecase. We agree an important direction for future work is exploring the extent to which our findings transfer to other44 settings. A concrete next step is evaluating model similarity on data from Kaggle competitions, which includes45 adiversesetofsample sizes, model classes, anddata types.