ParameterTuning

Then, we can use the data not used ineach stage toevaluate the out-of-sample performance ofthe other stage. Assumption 4. For each a A, the operator Ea is compact with singular system TheoperatorEa (denoted byKx in [20]) is defined by the relevant densities accordingly (see the paragraph after Lemma 2 of [20]). It is easy to see that Assumptions 4 and 5 are required for using Proposition 5. Remark 2. The difference between the first condition in Assumption 2 and Condition 3 in [20] is in the approach to establishing that the conditional expectationE[Y|A=a,Z = ] belongs to N(Fa) .Morespecifically,Condition 3in[20]isequivalent tohavingN(Fa)={0}andsoany nontrivialL2(PZ|A=a)-function is in the orthogonal complement. Lemma 2. Under Assumptions 1, 2, 4 and 5, for eacha A, there exists a function h a By Lemma 1, the regression functionE[Y|A=a,Z = ] is in N(E a) . For simplicity, we set all regularization terms to zero.