Planning for Markov Decision Processes with Sparse Stochasticity