[D] Are Model Sizes Approaching Human Cortical Numbers?

As someone with a neuroscience background, it's interesting to see the parallels between the parameter size & number of neurons in the recent GPT-3 unveiling versus the cortical regions of the brain responsible for language processing. Estimates put cortical neurons at 25B, and there is somewhere on the order of 7K connections between each neuron (further pruned over time), putting the total number of cortical parameters at somewhere close to 175T. Let's say language processing areas are 1/5th of this, the rest being dedicated to vision, higher order processing, executive function, etc. This puts the language generating portion of the human brain at maybe 35T parameters - yet we're capable of producing such fantastic results with GPT-3's 175B, a number which is 1/200th of the size. Of course, the two are not directly equatable - GPT-3, for example, has'read' several million books' worth of content at this point, whereas the average human has read just a few dozen by the time they're able and language-producing, but it's still interesting to reason about.