Aspects of human memory and Large Language Models

Large Language Models (LLMs) are huge artificial neural networks which primarily serve to generate text, but also provide a very sophisticated probabilistic model of language use. Since generating a semantically consistent text requires a form of effective memory, we investigate the memory properties of LLMs and find surprising similarities with key characteristics of human memory. We argue that the human-like memory properties of the Large Language Model do not follow automatically from the LLM architecture but Figure 1: Recall accuracy for a serial memory are rather learned from the statistics of the experiment with human subjects (sample training textual data. These results strongly data from [4]) and for a memorization experiment suggest that the biological features of human of a list of 20 facts of the has-a type for memory leave an imprint on the way that we the Large Language Model GPT-J [5] studied structure our textual narratives.