The Price of Diversity in Assignment Problems