Instance-wise Supervision-level Optimization in Active Learning

For classification tasks, a weak supervision approach is often designed to just attach rough Active learning (AL) is a label-efficient machine learning class labels to individual instances. For example, instead paradigm that focuses on selectively annotating high-value of attaching the exact class label "song sparrow" or "house instances to maximize learning efficiency. Its effectiveness sparrow" to an image instance, annotators can attach a can be further enhanced by incorporating weak supervision, rough class label "sparrows." This approach drastically reduces which uses rough yet cost-effective annotations instead annotation costs (budgets) because the fee paid to a of exact (i.e., full) but expensive annotations. We introduce non-expert annotator, who can only assign rough class labels a novel AL framework, Instance-wise Supervision-rather than exact ones via a crowdsourcing service, is Level Optimization (ISO), which not only selects the instances lower than that of an expert annotator. Hereafter, we call to annotate but also determines their optimal annotation rough classes as superclasses; therefore, "sparrows" is a superclass level within a fixed annotation budget.