Sleep-Like Unsupervised Replay Improves Performance when Data are Limited or Unbalanced

Anthony Bazhenov1,, - Pahan Dewasurendra1,, - Giri Krishnan2, and - Jean Erik Delanois2,3 1Del Norte High School, 16601 Nighthawk Ln, San Diego, CA 92127 2Department of Medicine, University of California, San Diego 3Department of Computer Science & Engineering, University of California, San Diego 9500 Gilman Drive, La Jolla, CA 92092 ABSTRACTDuring sleep, replay of recently learned memories along with relevant old memories enables the network to form The performance of artificial neural networks (ANNs) stable long-term memory representations [5] and reduces degrades when training data are limited or imbalanced. The idea of replay In contrast, the human brain can learn quickly from justhas been explored in machine learning to enable contina few examples. Here, we investigated the role of sleep inual learning. However, spontaneous unsupervised replay improving the performance of ANNs trained with limitedfound in the biological brain and implemented here is significantly different compared to explicit replay of pastdata on the MNIST and Fashion MNIST datasets. Sleep inputs implemented in machine learning rehearsal meth-was implemented as an unsupervised phase with local Hebbiantypelearningrules.