As inthe classical problem, weights are fixed by an adversary and elements appear in random order. In contrast to previous variants of predictions, our algorithm only has access toamuch weakerpiece ofinformation: anadditive gapc.

As such we see that this variance depends on the structure of the densityρX with the variance of (I +λL) 1δX, and the labelling noise with the variance of(Y |X).

Wewill also writeζθ(x) = fθ(x) f?(x)to denote the fitting error. We use Gaussian initialization: if` {1,2,...,L}, the weights are initialized as

By contrast, there isonly one quadratic programming problem solving inour proposed method, which significantly reduces time complexity, especially when the number of particles is high. The mean square error for each task and the average results are shown in Table 1. MT-SGD outperforms thesecond-best method, MOO-SVGD, with0.2251vs. However, on the one hand, computingU's entries can be accelerated in practice bycalculating theminparallel sincethereisnointeraction between themduring forwardpass. Allimagesareresizedto 64 64 3. Due tospace constraints, we report only the abbreviation ofeach task inthe main paper,their full namesarepresentedbelow.

Let t(s, a)= Q(s, a) ˆQ (s, a)andFt(s, a)= rpeer+ maxb2 AQ(s0,b) ˆQ (s, a). In(A4), we robust DQNalgorithmwithpeersampling, inwhichtheoriginlossis`((s, a), y), also calibrated.

Combining Lemma Assume H1) holds, N gr P n 9i N s.t.|Xi|2 2 ln,

Supposeσ(θ,z) Rd,we define the data-dependent featureby φMatrix(θ)= h σ(θ,z(1)),...,σ(θ,z(m)) i> Rm d, Notice that the score of most neuron can be calculated cheaply usingqi,A(k) and gi,A(k). The only exception are neuron withai(k) > 0 and γi,A(k) < ai(k)/(1 ai(k)). Andthenwegenerate thetraining data by sampling featurex from Unif[0,1] (each coordinate is sampled independently) and then generatelabely = Fgen(x). Wealso include the pruned model using global imitation,whichisdenotedas Fnglobal. Following the layer-wise procedure introduced in Section 2.4, suppose that the algorithms prunes F` to f`,A`, ` [L].

Convergence of stochastic gradient descent for non-smooth problems is a known result. For completeness, wereproduce and adapt ausual proof toour setting. Let us denote byF the class of functions fromX toY we are going to work with. Assumption 1 states that we have a well-specified modelF to estimate the median,i.e. Let us begin by controlling the estimation error.

We study theregretincurred bytheagent, firstwhen sheknowsherrewardfunction but does not know the distribution of the task duration, and then when she does not knowher reward function, either.

In most applications,E " Y to begin with (ally are potential maximizer for some vector of costs, otherwise theyare not included in the set), and all points inY havepositive mass. Ittherefore also satisfies this property. We recall that we assume thatθ yields a unique maximum to the linear program onC. As a consequence, all convergent subsequences ofyn converge to the same limity pθq: it is the unique accumulation point of this sequence.Itfollowsdirectlythat yn convergesto y pθq,asitlivesinacompactset,whichyieldsthe desired result. Using different reference vectorsv yield different perturbed operations, andv " p1,2,...,dq is commonlyused.