Contrastive Self-Supervised Learning at the Edge: An Energy Perspective

Abstract--While contrastive learning (CL) shows considerable promise in self-supervised representation learning, its deployment on resource-constrained devices remains largely underexplored. The substantial computational demands required for training conventional CL frameworks pose a set of challenges, particularly in terms of energy consumption, data availability, and memory usage. We conduct an evaluation of four widely used CL frameworks: SimCLR, MoCo, SimSiam, and Barlow Twins. We focus on the practical feasibility of these CL frameworks for edge and fog deployment, and introduce a systematic benchmarking strategy that includes energy profiling and reduced training data conditions. Our findings reveal that SimCLR, contrary to its perceived computational cost, demonstrates the lowest energy consumption across various data regimes. Finally, we also extend our analysis by evaluating lightweight neural architectures when paired with CL frameworks. Our study aims to provide insights into the resource implications of deploying CL in edge/fog environments with limited processing capabilities and opens several research directions for its future optimization. Over the years, a variety of contrastive learning (CL) approaches have been developed, including popular frameworks such as SimCLR [1], MoCo [2], BYOL [3], SimSiam [4], and Barlow Twins [5], each offering specific advantages and trade-offs. These frameworks aim to learn representations by distinguishing between similar (positive) and dissimilar (negative) samples in a latent space. While some methods rely on large negative sample sets to achieve high-quality representations, others bypass the need for negative pairs through momentum encoders or predictor networks.