CLIPGraphs: Multimodal Graph Networks to Infer Object-Room Affinities