Simple is Better and Large is Not Enough: Towards Ensembling of Foundational Language Models