Speeding up Policy Simulation in Supply Chain RL