An Embarrassingly Simple Approach to Enhance Transformer Performance in Genomic Selection for Crop Breeding