A Primal-Dual Framework for Transformers and Neural Networks