Revisiting Audio-language Pretraining for Learning General-purpose Audio Representation