Game of Gradients: Mitigating Irrelevant Clients in Federated Learning