Affordance Field Intervention: Enabling VLAs to Escape Memory Traps in Robotic Manipulation