Soundwave: Less is More for Speech-Text Alignment in LLMs