Analyzing the Role of Semantic Representations in the Era of Large Language Models