Review for NeurIPS paper: On Numerosity of Deep Neural Networks

This paper demonstrates that an analysis relied upon in a previous paper (Nasr et al., 2019) to identify number-sensitive units in a neural network trained for object recognition is flawed, and that indeed the same network with randomly initialized weights also has a large number of number sensitive units. Moreover, the number of units detected depends strongly on the sample size of the statistical test, with larger sample sizes detecting no number sensitive units. The paper additionally performs some analyses on a network trained specifically to predict number. The reviewers generally felt that the demonstration of Nasr et al.'s flawed analysis was important, with R2 arguing that the work is "imperative to publish" and R1 and R3 finding the experiments in the first part of the paper convincing. However, R1, R3, and R4 all had concerns with the second part of the paper, in which it is claimed that a network trained to classify number (Nu-Net) can learn to subitize. I feel that the results in the first part of the paper are sufficiently impactful that the paper should be accepted.