AT ask Level Case Study

This section illustrates how a model's performance may vary across different tasks associated with We analyzed the performance of Llama-3-Instruct-70B on the new term "wokely," The book's cover was described as wokely by several reviewers. A. it struggled to attract attention on the bookstore displays despite a B. many readers were enticed to buy it, strengthening its presence on C. readers were intrigued and the book's sales experienced an unexpected surge worldwide. D. the publisher decided to release a limited edition with a special In the previous sentence, does _ refer to A. Is this example in line with commonsense and grammatically correct? As observed, the model only answered correctly in the COMA task but failed in the other two tasks. In the COMA task, the model successfully inferred that "wokely" carries a negative connotation, Although the phrase "hard to find a satisfying These results provide a comprehensive evaluation of the model's understanding of the term "wokely."