When Do Program-of-Thoughts Work for Reasoning?