Code Driven Planning with Domain-Adaptive Critic