Such Swapped errors are also common vehicles, object detection remains fairly in many classification datasets (Northcutt et al., 2021a), brittle in part due to annotation errors that plague but the increased complexity of object detection annotation most real-world training datasets. We propose introduces potential for more varied types of label errors ObjectLab, a straightforward algorithm to detect than encountered in classification. We propose an algorithm, diverse errors in object detection labels, including: ObjectLab, that utilizes any trained object detection model overlooked bounding boxes, badly located boxes, to estimate the incorrect labels in such a dataset, regardless and incorrect class label assignments. Object-which of these 3 types of mistake the data annotators made. Lab utilizes any trained object detection model to score the label quality of each image, such that Training and evaluating models with incorrect bounding box mislabeled images can be automatically prioritized annotations is clearly worrisome.

Real-world data for classification is often labeled by multiple annotators. For analyzing such data, we introduce CROWDLAB, a straightforward approach to utilize any trained classifier to estimate: (1) A consensus label for each example that aggregates the available annotations; (2) A confidence score for how likely each consensus label is correct; (3) A rating for each annotator quantifying the overall correctness of their labels. Existing algorithms to estimate related quantities in crowdsourcing often rely on sophisticated generative models with iterative inference. CROWDLAB instead uses a straightforward weighted ensemble. Existing algorithms often rely solely on annotator statistics, ignoring the features of the examples from which the annotations derive. CROWDLAB utilizes any classifier model trained on these features, and can thus better generalize between examples with similar features. On real-world multi-annotator image data, our proposed method provides superior estimates for (1)-(3) than existing algorithms like Dawid-Skene/GLAD.