Manifold-Preserving Transformers are Effective for Short-Long Range Encoding