Selective Test-Time Adaptation for Unsupervised Anomaly Detection using Neural Implicit Representations