Thestandard supervised learning paradigm workseffectivelywhentraining data shares the same distribution as the upcoming testing samples.

After prediction, the true labelyt is revealed and the lossℓ(yt,ˆyt)is incurred.

Therefore, if ν() < +, then we can bound(10) with eαν(). To avoid crowded notations, we drop the conditioning onz from Pr[ |ρ = z]. The issue is how to proceed. Let φ be the standard normal density function andΦ be the CDF. The algorithm using SVT suchthat itonly releases the private answerstothe queries if the answer is sufficiently different from the "guess".

Inthis section, we provide theoretical analysis ofHSPG. Moreover, we further point out that: (1) theSub-gradient Descent Stepwe used to achieve a "close enough" solution canbereplaced byothermethods, and(2)theAssumption 4isonlyasufficientcondition thatwecouldusetoshowthe"closeenough"condition. B.1 RelatedWork Problem (12)has been well studied indeterministic optimization with various algorithms that are capable ofreturning solutions with both lowobjectivevalueandhigh group sparsity under proper λ(95;73;42;64). For example, proximal stochastic variance-reduced gradient method (Prox-SVRG)(88)and proximal spider (Prox-Spider) (97) are developed to adopt multi-stage schemes based on the well-known variance reduction technique SVRG proposed in (46) and Spider developed in (22) respectively. Under Assumption 1, the search directiondk is a descent direction forψBk(xk), i.e., d>k ψBk(xk)<0.

A is given byAk,`,Pr[yk(x)=`], which represents the probability ofkth service producing label `. The scalar function Fk,`(X), Pr[qk(x) X|y(x) = `] is the probability of the produced quality score from thekth service less than a thresholdX conditional on that its predicted label is`. There are 3 steps for solving problem 3.3. Thus,ψi() and ψi,j() are piece wise quadratic functions. To solve Problem 3.2, let us first denoteΩ3 = {x In other words,z0 has the same objective function value asz .

In the sequel, we empirically show the effect of different numbers of local updates on the fixed point. We consider cases withK = 1, K = 10, K = 20, K = 50. From Assumption 1, it is obvious thatgi(x,y) is convex-concave. Then, we conclude that there exists someη1 > 0 such that h(η) > 0, 0 < η < η1.