Analyzing Examinee Comments using DistilBERT and Machine Learning to Ensure Quality Control in Exam Content

To ensure that the items are of sufficient quality to be included in the test, multiple rounds of item review are conducted both before and after the test is administered. Typically, once the testing period has ended, psychometricians will analyze the response data using var ious methods to identify any items that require further review based on their statistical properties (e.g., p - value, point - biserial correlation, etc.). For example, one item with a low point - biserial correlation value can be flagged for further review due to poor discrimination. While flagging items using their statistics can help identify potentially problematic items, it does not guarantee that the flagged items actually contain issues. Therefore, subject matter experts (SMEs) need to review the flagged items to determine whether they indeed pose any problems.