Answer-Centric or Reasoning-Driven? Uncovering the Latent Memory Anchor in LLMs