From Instructions to Constraints: Language Model Alignment with Automatic Constraint Verification