On the Effectiveness of Speech Self-supervised Learning for Music