Capturing Symmetry and Antisymmetry in Language Models through Symmetry-Aware Training Objectives