Communication compression isacrucial technique formodern distributedlearning systems to alleviate their communication bottlenecks over slower networks.

A unified learning theory for arbitrarily fast computation with arbitrarily slow neurons

While large language models have demonstrated this ability, how in-context learning could be performed over graphs is unexplored. In this paper, we develop Pr etraining O ver D iverse I n-Context G raph S y stems (PRODIGY), the first pretraining framework that enables in-context learning over graphs.