Symmetry Leads to Structured Constraint of Learning