Efficient Test-Time Retrieval Augmented Generation