SimSiam in PyTorch, Part 1: The Data

In this series, we will write, in PyTorch, SimSiam, an awfully simple yet competitive self-supervised learning (SSL) technique that dispenses with complicated tricks prevalent in other SSL algorithms like BYOL's momentum encoder, SimCLR's negative pairs, and SwAV's online clustering. Audience: Because SimSiam is quite straightforward, there are no prerequisites concerning self-supervised learning, and everything will be covered or helpful articles are linked. However, the fundamentals of convolutional neural networks plus PyTorch are required. Without further ado, let's get coding! Oft-times, I prefer top-down approaches for diving into a topic compared to starting with the building blocks and gradually building up to the bigger idea.