Robust Control with Gradient Uncertainty