eP-ALM: Efficient Perceptual Augmentation of Language Models