The table ( right) shows 100 epoch results using best lr and wd values found at 50 epochs. ViT's patchify stem differs from the proposed convolutional stem in the type of convolution used and We investigate these factors next. The focus of this paper is studying the large, positive impact of changing ViT's default We use AdamW for all experiments. Figure 7 shows the results. The table ( right) shows 100 epoch results using optimal lr and wd values chosen from the 50 epoch runs.

This large-kernel plus large-stride convolution runs counter to typical design choices of convolutional layers in neural networks.

Influence functions (IF) have been seen as a technique for explaining model predictions through the lens of the training data.

Neural Information Processing Systems http://nips.cc/

Ouranalysishelps13 illustrate why this worst-case behavior does not attain in practice. The standard union bound does not take advantage of model similarity; the refinement in equation (3) is the24 basis for our calculations that highlight the beneficial effect of model similarity. Our bounds demonstrate a link between similarity and protection against overfitting in both the30 non-adaptiveandadaptivecase. We agree an important direction for future work is exploring the extent to which our findings transfer to other44 settings. A concrete next step is evaluating model similarity on data from Kaggle competitions, which includes45 adiversesetofsample sizes, model classes, anddata types.

In addition to solving (1), we also wish topreserve privacyfor the people who contributed to the dataset Z.