Robust Bayesian Scene Reconstruction by Leveraging Retrieval-Augmented Priors