We find that Lookahead is less sensitive to suboptimal hyperparameters and therefore lessens the need for extensive hyperparameter tuning.

However, such models are difficult to meaningfully interpret and check for correctness. Thus, researchers have tried to understand the behavior of such networks.

Neural Information Processing Systems http://nips.cc/

In this section, we provide the details and hyperparameters for SwA V pretraining and transfer learning. A.1 Implementation details of SwA V training First, we provide a pseudo-code for SwA V training loop using two crops in Pytorch style: # C: prototypes (DxK) # model: convnet + projection head # temp: temperature for x in loader: # load a batch x with B samples x_t = t(x) # t is a random augmentation x_s = s(x) # s is a another random augmentation z = model(cat(x_t, x_s)) # embeddings: 2BxD scores = mm(z, C) # prototype scores: 2BxK scores_t = scores[:B] scores_s = scores[B:] # compute assignments with torch.no_grad(): The latter methods require sharing the feature matrix across all GPUs at every batch which might become a bottleneck when distributing across many GPUs. A.2 Data augmentation used in SwA V We obtain two different views from an image by performing crops of random sizes and aspect ratios. We use the 1% and 10% splits specified in the official code release of SimCLR.