SAVANT: Semantic Analysis with Vision-Augmented Anomaly deTection