A survey of mixed-precision neural networks

In their paper Mixed-Precision Neural Networks: A Survey, Mariam Rakka, Mohammed E. Fouda, Pramod Khargonekar and Fadi Kurdahi have reviewed recent frameworks in the literature that address mixed-precision neural network training. Here, they tell us more about mixed-precision neural networks and the main findings from their survey. Mixed-precision neural networks are neural networks with varying precision (i.e., bitwidth allocation) across layers, kernels or weights. They are now gaining momentum as the need for energy-efficient and high throughput AI hardware is growing. Binary neural networks are considered the most efficient to be deployed on hardware, however, they exhibit a non-negligible drop in the model accuracy compared to floating-point neural networks which give the best accuracy and worst energy and latency efficiency.