The capability of Large Language Models (LLMs) such as GPT-4 to engage in conversation with humans presents a significant leap in Artificial Intelligence (AI) development that is broadly considered to be disruptive for certain technological areas. A human interacting with an LLM may indeed perceive the LLM as an agent with a personality, to the extent that some have even called them sentient (De Cosmo, 2022). While we, of course, do not subscribe to the notion that LLMs are sentient - nor do we believe it is as yet clear what it means to even ask whether an LLM has a personality - there is still the appearance of agency and personality to the human user interacting with the system. Subjecting an LLM to psychometric tests is thus, in our view, less an assessment of some actual personality that the LLM may or may not have, but rather an assessment of the personality or personalities perceived by the human user. As such, our interest is not only in the actual personality profile(s) resulting from the tests, but also in the question whether the profiles are stable over re-tests and how they vary with different (relevant) parameter settings. At the same time, the results beg the question why the results of the tests are what they are.

A major challenge in Explainable AI is in correctly interpreting activations of hidden neurons: accurate interpretations would provide insights into the question of what a deep learning system has internally detected as relevant on the input, demystifying the otherwise black-box character of deep learning systems. The state of the art indicates that hidden node activations can, in some cases, be interpretable in a way that makes sense to humans, but systematic automated methods that would be able to hypothesize and verify interpretations of hidden neuron activations are underexplored. In this paper, we provide such a method and demonstrate that it provides meaningful interpretations. Our approach is based on using large-scale background knowledge approximately 2 million classes curated from the Wikipedia concept hierarchy together with a symbolic reasoning approach called Concept Induction based on description logics, originally developed for applications in the Semantic Web field. Our results show that we can automatically attach meaningful labels from the background knowledge to individual neurons in the dense layer of a Convolutional Neural Network through a hypothesis and verification process.