Conventional Federated Learning (FL) systems inherently assume a uniform processing capacity among clients for deployed models. However, diverse client hardware often leads to varying computation resources in practice.

As computing the inverse second-order derivatives is the most computation-intensive operation, we will focus on it. In Section 3.1, we use the trick of least square to compute the We can leverage the Neumann series to compute the matrix inverse. B.1 Proof of the Approximation by Implicit Gradients Here, we provide the proof for J. B.2 Proof of Theorem 3.1 Before we prove our main theorem, we prove several essential lemmas as below. Using Assumption 3.4 and 3.5 directly lead to r By Assumption 3.4, we have r By Lemma B.1 and Lemma B.2, we have r If Assumption 3.4 and 3.5 hold, then the The linear model we use is a matrix that maps the input data into a vector. LeNet model is a convolutional neural network with 4 convolutional layers and 1 fully connected layer.