Learning cosmology and clustering with cosmic graphs