789ba2ae4d335e8a2ad283a3f7effced-Supplemental.pdf

A is given byAk,`,Pr[yk(x)=`], which represents the probability ofkth service producing label `. The scalar function Fk,`(X), Pr[qk(x) X|y(x) = `] is the probability of the produced quality score from thekth service less than a thresholdX conditional on that its predicted label is`. There are 3 steps for solving problem 3.3. Thus,ψi() and ψi,j() are piece wise quadratic functions. To solve Problem 3.2, let us first denoteΩ3 = {x In other words,z0 has the same objective function value asz .