The manifold hypothesis presumes that high-dimensional data lies on or near a low-dimensional manifold. While the utility of encoding geometric structure has been demonstrated empirically, rigorous analysis of its impact on the learnabil-ity of neural networks is largely missing. Several recent results have established hardness results for learning feedforward and equivariant neural networks under i.i.d.

Relevant to this paper are examinations of the computational power of neural networks after training, i.e., the training process is not taken into account but instead the computational power of an optimally trained network is studied. Starting already in the nineties, the expressive power of feed-forward neural networks (FNNs) has been related to Boolean threshold circuits, see, e.g., [Maass et al., 1991, Siegelmann and Sontag, 1995,

V alence, Arousal and Emotion label, with annotations gathered using Prolific.

To mitigate the threat, existing solutions attempted to search for backdoor triggers, which can be time-consuming when handling a large search space.