Active Fairness in Algorithmic Decision Making