Towards Algorithmic Fairness in Space-Time: Filling in Black Holes