Robust Multi-Agent Decision-Making in Finite-Population Games