Can REF output quality scores be assigned by AI? Experimental evidence

This document describes strategies for using Artificial Intelligence (AI) to predict some journal article scores in future research assessment exercises. Five strategies have been assessed. These are summarised here for completeness, but we recommend that AI predictions are not used to help make scoring decisions yet but are further explored through pilot testing in the next REF or REF replacement. The pilot testing should assess whether using AI predictions and prediction probabilities alongside, or instead of, bibliometric data would be helpful for any UoAs. For example, depending on UoA, AI predictions may be used to help mop up difficult scoring decisions near the end of the assessment period, to gain interdisciplinary input, as a tiebreaker in the way that bibliometrics are currently sometimes used, or to cross check the final scores.