Towards a tailored mixed-precision sub-8-bit quantization scheme for Gated Recurrent Units using Genetic Algorithms

Despite the recent advances in model compression techniques for Model compression techniques such as quantization have been deep neural networks, deploying such models on ultra-low-power successfully applied to Convolutional Neural Networks (CNNs), embedded devices still proves challenging. In particular, quantization allowing them to be deployed on embedded devices with limited schemes for Gated Recurrent Units (GRU) are difficult to computational resources. Remarkably, the quantization of RNNs tune due to their dependence on an internal state, preventing them has not been explored as extensively, potentially due to the additional from fully benefiting from sub-8bit quantization. In this work, we complexity introduced by their recurrent nature. Among the propose a modular integer quantization scheme for GRUs where the most notable works, [1] propose binary, ternary, and quaternary bit width of each operator can be selected independently. We then quantization schemes for RNNs and evaluate it on sentiment analysis, employ Genetic Algorithms (GA) to explore the vast search space [11] combines structural pruning and 8-bit quantization to of possible bit widths, simultaneously optimizing for model size optimize LSTMs for speech enhancement on a Cortex-M7 embedded and accuracy. We evaluate our methods on four different sequential platform, [20] presents quantization schemes for the standard tasks and demonstrate that mixed-precision solutions exceed LSTM and its variants, based on fixed-point arithmetic, evaluating homogeneous-precision ones in terms of Pareto efficiency. Our them on speech recognition; finally [26] employs mixed-precision results show a model size reduction between 25% and 55% while FP16 and 8-bit integer quantization to deploy speech enhancement maintaining an accuracy comparable with the 8-bit homogeneous models based on LSTMs or GRUs on a RISC-V embedded target.