Attributing Fair Decisions with Attention Interventions