Google Brain's new super fast and highly accurate AI: the Mixture of Experts Layer.

One of the big problems in Artificial Intelligence is the gigantic amount of GPUs (or computers) needed to train large networks. The training time of neural networks grows quadratically (think squared) in function of their size. This is due to how the network is trained. For each example, the entire network is modified, even though some parts might not even activate while processing this particular example. However, the memory of a network is directly dependent on the size of the network.