Learning Broken Symmetries with Approximate Invariance