Learn To be Efficient: Build Structured Sparsity in Large Language Models