Can We Generate Visual Programs Without Prompting LLMs?