SNR = 10 log10 2 Figure 1: AWGNchannelwithnoisyfeedback (left). Figure 5: Noiseinfirstphasevs. firstparitybit (left) andsecondparitybit (right).

Inductive biases encoding known data symmetries are key to make deep learning models generalize in high-dimensional settings such as computer vision, speech processing and computational neuroscience, just to name a few.

Using a single projection vector, we then project these individual sub-tokens onto a one-dimensional subspace. Importantly, we notice that we can initialize this projection vector cheaply using first-order batch statistics andthen keepitfixedthroughout training. Wethen reconstruct the original tokens using the same vector during the backward pass.

However, the completion accuracy can be drastically different over different entries.

Neural Information Processing Systems http://nips.cc/

B.1 Learningdetails All of our code and experiments are implemented in Python using the PyTorch [5]deep learning framework.