Learning to Steer Markovian Agents under Model Uncertainty