On Affine Homotopy between Language Encoders