PyTorch Introduces Native Automatic Mixed Precision Training

With the increasing size of deep learning models, the memory and compute demands too have increased. Techniques have been developed to train deep neural networks faster. One approach is to use half-precision floating-point numbers; FP16 instead of FP32. Recently, researchers have found that using them together is a smarter choice. Mixed precision is one such technique, which can be used to train with half-precision while maintaining the network accuracy achieved with single precision.