An Empirical Analysis of Discrete Unit Representations in Speech Language Modeling Pre-training