Constructive Large Language Models Alignment with Diverse Feedback