Appendix AImplementationdetails

The encoder contains three linear layers with output size [d,k,k], each but the last layer is followed by batch normalization, witheps = 0.00005 and momentum=0.1,andtheReLU Thedecoder contains threelinear layers withoutput size [k,k,d] where each but the last layer contains a Batch normlization and the ReLu activation similar asabove. Following the standard linear evaluation procedure inself-supervised learning works (32;34),we used an one linear layer network as the linear decoder for the decoding accuracy. We used the neural activity dataset that is collected from two rhesus macaque monkeys (Chewie and Mihi). They were trained to move the computer cursor to reach a target on a screen.