Semantic Exploration from Language Abstractions and Pretrained Representations