Efficient infusion of self-supervised representations in Automatic Speech Recognition