Towards Resource Efficient and Interpretable Bias Mitigation in Large Language Models