Paralinguistics-A ware Speech-Empowered Large Language Models for Natural Conversation