The More is not the Merrier: Investigating the Effect of Client Size on Federated Learning