Speeding up Deep Learning with Quantization

In last week, Facebook has just open sourced their matrix multiplication library which you can read it here .Readers may quickly find the word "quantized" or "quantization" appear a lot in that article and wonder what is magical about this new hype word that help giving 2.4x performance boost on CPU. I'm going to give some beginner's introduction to quantization, I may use some simple maths a long the way but don't worry, I promise it is very gentle. Recent advancement in AI or more specifically a technique called deep learning (DL) brought a lot of excitement about the type of applications that are possible e.g. Despite of many exciting news about breakthrough in research or applications, the fact is that we still don't see a lot of products or services (yet, but it is changing rapidly). One of the reasons is that it is costly to run deep neural network which often require expensive and power hungry GPU (graphical processing unit) to run.