ChatGPT better than undergraduates at solving SAT problems, study suggests

ChatGPT can solve problems at a level that matches or surpasses an undergraduate student, according to a new study. Researchers found that the GPT-3 large language model that underpins the chatbot performed about as well as US college undergraduates when asked to solve reasoning problems that appear on intelligence tests or exams such as the American college admission test, the SAT. Psychologists at the University of California, Los Angeles tested GPT-3's ability to predict the next image in a complex array of shapes, after converting the images to a text format that the model could process and also ensuring the model would never have encountered the questions before. The same problems were put to 40 UCLA undergraduates and the researchers found that GPT-3 solved 80% of the problems correctly, well above the average score of just below 60% for the human participants. The researchers also prompted the model to solve some SAT "analogy" questions – selecting pairs of words that are linked in some way – that they believe had not been published on the internet and therefore could not have appeared in the vast amount of data it was trained on.