Chemical Language Model Linker: blending text and molecules with modular adapters