Rethinking Byzantine Robustness in Federated Recommendation from Sparse Aggregation Perspective