PAPERCLIP: Associating Astronomical Observations and Natural Language with Multi-Modal Models