The Cosmic Graph: Optimal Information Extraction from Large-Scale Structure using Catalogues