Appendix

A Extended Related Work There are two groups of approaches to debias click data for ranking. The first is to model user's behavior to infer relevance from biased click signals, known as click models [16, 21, 20, 9, 8]. However, most click models focus on predicting clicks, rather than optimizing the ranking performance [29, 4]. They are usually separated from the L TR frameworks, and the relevance inference is an afterthought [4]. The second group tries to directly learn unbiased ranking models from biased clicks, known as unbiased learning to rank (UL TR).