Reviews: Heterogeneous Bitwidth Binarization in Convolutional Neural Networks

The paper makes an observation that in general 2-bit models are more accurate than 1-bit model. The goal in this paper is to make the performance of lower-bit models closer to than of without (significantly) compromising on their efficiency. The paper achieves this by binarizing to a mixture of bitwidths as opposed to the traditional approach of a uniform bitwidth to all values. In particular, it extends the approach presented in Tang et al. (2017) that homogeneously binarizes to n bits. The paper proposes to examine individual values and decide whether to binarize them (using the Mask Tensor M).