Multiplication-Free Transformer Training via Piecewise Affine Operations

Neural network training consists largely of matrix multiplications that generally account for the vast majority of the computational cost for standard architectures such as transformers.