However, existing quantization methods usually have inferior performancewhenusedonLSTMs.

In this work, we investigate training methods for quantized neural networks from a theoretical viewpoint.

Neural Information Processing Systems http://nips.cc/

The long-short-term memory (LSTM), though powerful, is memory and computation expensive.

Adding normalization makes the analysis even more non-trivial.

I mostly like very much the fact that you can train with binary weights, in contrast to previous works. Note that is FPGA section should be improved. It should be made more concrete (showing at least a diagram how the weights are placed and how the data are routed through the network. Specifically, how to route the convolutional layers) or removed.

Deep Neural Networks (DNN) have achieved state-of-the-art results in a wide range of tasks, with the best results obtained with large training sets and large models. In the past, GPUs enabled these breakthroughs because of their greater computational speed. In the future, faster computation at both training and test time is likely to be crucial for further progress and for consumer applications on low-power devices. As a result, there is much interest in research and development of dedicated hardware for Deep Learning (DL). Binary weights, i.e., weights which are constrained to only two possible values (e.g.