The Process of Categorical Clipping at the Core of the Genesis of Concepts in Synthetic Neural Cognition

The result of this categorical segmentation is reflected in the creation, by each neuron, of a synthetic category of thought, a concept, or, to put it differently, a categorical dimension carried by this neuron [101, 102]. This synthetic conceptual category is, among other things, defined by its extension, that is, the set of tokens for which the neuron associated with this category is (sufficiently) activated. In a previous work [105], we investigated the mathematical-cognitive factors of categorical segmentation performed by the synthetic neurons of language models. In this preliminary exploratory study, we examined, both quantitatively and qualitatively, the genetic elements influencing this categorical segmentation. Based on the aggregation function 1 Σ(w i,jx i,j) + b, which partly governs this cognitive process, we identified three key causal elements of a mathematical and cognitive nature involved in this conceptual partitioning process. First, the "x effect" or synthetic categorical priming, which refers to the fact that the activation of the categories carried by precursor neurons in layer n affects the activation of the categories specific to their associated target neurons in layer n + 1, thereby directly impacting their categorical extension. In other words, the more a token belongs to the extension of a precursor category in layer n (i.e., the more this token is activated in the involved neuron), the greater its potential to belong to the extension of its superordinate category (i.e., its potential activation in the neuron of layer n +1). This phenomenon of categorical priming thus partly governs the categorical segmentation performed in layer n + 1, that is, the determination of the subset of tokens constituting the categorical extension of the concepts carried by the neurons in layer n + 1. Second, the "w effect" or synthetic categorical attention, which relates to the fact that the weights of the connections between a target neuron (layer n + 1) and its precursor neurons (layer n) govern the degree of relevance attributed to the precursor categories in constructing the categorical segment of their corresponding target neurons.