9bcd0bdb2777fe8c729b682f07e993f1-Supplemental-Datasets_and_Benchmarks.pdf

MIRcontains25uniquelabels,andweremoved the label "night" as it is not in the label set of any MLAPIs. For each instance in those datasets, we have evaluated the prediction from the mainstream ML APIs from 2020 to 2022. HAPI was collected from 2020 to 2022. For classification tasks, the predictions/annotations of each API were collected in the spring of 2020, 2021, and 2022. Theoriginal IMDB dataset hasbeenpartitioned into training and testing splits, and thus we used its testing split, including 25,000 textparagraphs.