New Neural Network with 500 Billion Parameters

Google just published a research article about its Pathways Language Model (PaML), a neural network with 500 billion parameters. It is unclear to me how many layers and how many neurons (also called nodes) it can handle. A parameter in this context is a weight attached to a link between two connected neurons. So the number of neurons is at most 500 billion, but it is most likely much smaller. By contrast, the average human brain has 86 billion neurons.