Distributed Associative Memory via Online Convex Optimization

ABSTRACT An associative memory (AM) enables cue-response recall, and associative memorization has recently been noted to underlie the operation of modern neural architectures such as Transformers. This work addresses a distributed setting where agents maintain a local AM to recall their own associations as well as selective information from others. Specifically, we introduce a distributed online gradient descent method that optimizes local AMs at different agents through communication over routing trees. Our theoretical analysis establishes sublinear regret guarantees, and experiments demonstrate that the proposed protocol consistently outperforms existing online optimization baselines. Index T erms-- Associative Memory, Distributed Optimization, Online Convex Optimization 1. INTRODUCTION An associative memory (AM), a classical concept in cognitive science, stores cue-response associations, recalling the response when the corresponding cue is presented [1]. This principle, fundamental to human cognition, provides a natural abstraction for modeling how information can be efficiently retained, updated, and retrieved.