Decision Tree Design for Classification in Crowdsourcing Systems

In recent work on classification in crowdsourcing systems, complex questions are often replaced by a set of simpler binary questions (microtasks) to enhance classification performance [1]-[4]. This is especially helpful in situations where crowd workers lack expertise for responding to complex questions directly. Each worker is given the entire set of questions in a batch mode and the workers provide their responses in the form of a vector. These binary questions can be posted as "microtasks" on crowdsourcing platforms like Amazon Mechanical Turk [5]. To improve classification performance in crowdsourcing systems, most of the works in the literature focus on enhancing the quality of individual tests, by designing fusion rules to combine decisions from heterogeneous workers [1]-[4], [6], [7], and by investigating the assignment of different tests to different workers depending upon their skill level [8], [9].