Sleep Deprivation in the Forward-Forward Algorithm

This paper aims to explore the separation of the two forward passes in the Forward-Forward algorithm from a biological perspective in the context of sleep. We show the size of the gap between the sleep and awake phase influences the learning capabilities of the algorithm and highlight the importance of negative data in diminishing the devastating effects of sleep deprivation. The Forward-Forward (FF) algorithm (Hinton, 2022) introduces a new learning procedure that provides a feasible model of how learning works inside the cortex. In contrast with backpropagation (Rumelhart et al., 1986), which has been previously shown to be an implausible explanation for learning in the brain (Lillicrap et al., 2020), the Forward-Forward algorithm aims to avoid the large memory footprint and overhead computation arising from the backward pass by introducing two separate forward passes that optimize opposite objectives. During training, one forward pass operates on real or positive data, while the other uses negative data, which can be generated internally by the network through top-down connections or supplied externally.