A Brain-to-Population Graph Learning Framework for Diagnosing Brain Disorders