Assessment Methods

Machine Learning in Psychometrics: Old News? Online Testing, Educational Assessment, Computerized Adaptive Testing Assessment Systems


In the past decade, terms like machine learning, artificial intelligence, and data mining are becoming greater buzzwords as computing power, APIs, and the massively increased availability of data enable new technologies like self-driving cars. However, we've been using methodologies like machine learning in psychometrics for decades. So much of the hype is just hype.

Towards artificial intelligence-based assessment systems


The application of AI to education has been the subject of academic research for more than 30 years, with the aim of making "computationally precise and explicit forms of educational, psychological and social knowledge which are often left implicit"3. The evidence from existing AI systems that assess learning as well as provide tutoring is positive with respect to their assessment accuracy4. AI is a powerful tool to open up the'black box of learning', by providing a deep, fine-grained understanding of when and how learning actually happens. In order to open this black box of learning, AI assessment systems need information about: (1) the curriculum, subject area and learning activities that each student is completing; (2) the details of the steps each student takes as they complete these activities; and (3) what counts as success within each of these activities and within each of the steps towards the completion of each activity. AI techniques, such as computer modelling and machine learning, are applied to this information and the AI assessment system forms an evaluation of the student's knowledge of the subject area being studied.