Both initialization methods ondownstream tasks canachievesimilar performance, butinitializing from BERT-base33 reduces the number of learning steps. In order to shorten the training time of our large-size model, we initialize34 it from BERT-large. We will also release a model trained from scratch. We further trained BERT-large using the35 same hyper-parameters, buttheresulted model didn'tsignificantly improvedownstream tasks compared tooriginal36 BERT-large.

The result is shown in Figure I.

We thank all the reviewers for their valuable and positive feedback. Thus we believe it is significant to study in this15 field, which isalsosupported bytheother reviewers. After double-checking, we do have the appendix attached in the supplementary material (in25 "N_Gram_Graph_Paper.pdf").

There is no obvious way to apply them to the datasets we consider.

Therefore, thepairwise term (refer toEq.6)

Numerosity cognition may well be a holistic mechanism. Like this reviewer we had the same urge to challenge the13 methodology and results of numerosity studies in neuroscience [21][22], but refrained from debunking directly the14 workspublishedintopjournals.