Appendix

AExtendedRelatedWork There are two groups of approaches to debias click data for ranking. To address it, Agarwal et al. [3] and Fang et al. [17] proposed to do intervention harvest by exploiting click logs with multiple ranking models. Nevertheless, they have a relatively narrow scope of application due to the strict assumption to construct interventional sets. Recently, some researchers proposed to jointly estimate relevance and bias [43, 4, 23, 25, 14]. Similar to them, our proposed method could jointly train the ranking model and observation model without intervention. However, these types of work mainly model the observation bias at a group level and ignore the characteristicsofeachdocument.