CIFAR-10 and MNIST datasets when using maximum mean discrepancy (MMD) with learned instead of fixed features. For a related method and in a similar spirit, Santos et al. (2019) show that for image

Thanks totheir scalability and lowmemory requirements, first-order methods are especially popular inthis setting (Bottouetal., 2018).

Random Reshuffling (RR) is an algorithm for minimizing finite-sum functions that utilizes iterative gradient descent steps in conjunction with data reshuffling. Often contrasted with its sibling Stochastic Gradient Descent (SGD), RR is usually faster in practice and enjoys significant popularity in convex and non-convex optimization. The convergence rate of RR has attracted substantial attention recently and, for strongly convex and smooth functions, it was shown to converge faster than SGD if 1) the stepsize is small, 2) the gradients are bounded, and 3) the number of epochs is large.

We give a proof in Appendix A.5. As we discuss in Appendix A.6, we can use this result to obtain First, by Lemma 3.4, we have F Then, by Lemma A.1, with probability at least 1, we have F Note that to show the claim, it suffices to show that for sufficient large, we have ( / 2) c ( s) ( s)+ ( 8 s) . The claim follows by taking the limit k!1 . We first prove a bound on the concentration of the empirical CDF to the true CDF. We proceed by bounding the two terms in the summation.

Firstly, many existing attribution techniques require either computing the gradients [58] or multiple queries to the model [19], both of which are slow and computationally expensive. In contrast, ICL is often applied inreal-time to a large foundation model [12] that necessitates the attribution approaches for ICL to be fast and efficient.

This scaling behavior coincides with the one reportedinTripuranenietal. (2020)usingGaussiancomplexity.