Emergent Computations in Trained Artificial Neural Networks and Real Brains

New computational techniques [1, 2, 3, 4, 5, 6, 7] enable neural networks to be trained on tasks similar to those used in experiments with behaving animals [8, 9, 10, 11, 12, 13, 14, 15]. Before these techniques became available, a researcher would hypothesize what computations the network should perform to execute the task, and build a network architecture capable of carrying them out. Then, numerical simulations of the model or mean field approximations allowed verifying whether the proposed network model performed the task as desired. This is unsatisfactory, as it does not allow identifying how a neural network could solve these tasks; the models thus constructed only reflect the researcher's intuitions about how the tasks could be performed. In contrast, trained networks provide us with a valuable tool to investigate mechanisms that networks could use to perform the tasks [16, 17, 18, 19, 6, 7, 20, 13, 21, 22, 23].