Reference-Specific Unlearning Metrics Can Hide the Truth: A Reality Check