Prompting Large Language Models with Audio for General-Purpose Speech Summarization