Review for NeurIPS paper: Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts

I was expecting a billion trillion scale evaluation and surprising number in the end because that was the whole motivation. It is very disappointing to see this kind of poor evaluations in a paper with such a good start.