Universal Speech Token Learning via Low-Bitrate Neural Codec and Pretrained Representations