Task Aware Modulation using Representation Learning: An Approach for Few Shot Learning in Heterogeneous Systems