Speech Retrieval-Augmented Generation without Automatic Speech Recognition