Understanding the Effect of using Semantically Meaningful Tokens for Visual Representation Learning