Allocation Requires Prediction Only if Inequality Is Low