Microsoft unveils Brainwave, a system for running super-fast AI

Microsoft made a splash in the world of dedicated AI hardware today when it unveiled a new system for doing high-speed, low-latency serving of machine learning models. The company showed off a new system called Brainwave that will allow developers to deploy machine learning models onto programmable silicon and achieve high performance beyond what they'd be able to get from a CPU or GPU. Researchers at the Hot Chips conference in Cupertino, California showed a Gated Recurrent Unit model running on Intel's new Stratix 10 field programmable gate array (FPGA) chip at a speed of 39.5 teraflops, without batching operations at all. The lack of batching means that it's possible for the hardware to handle requests as they come in, providing real-time insights for machine learning systems. The model that Microsoft chose is several times larger than convolutional neural networks like Alexnet and Resnet-50, which other companies have used to benchmark their own hardware.