Do Language Models Refer?