Contrastive Geometric Learning Unlocks Unified Structure- and Ligand-Based Drug Design