Open-World Object Manipulation using Pre-trained Vision-Language Models