Gradient Estimation for Unseen Domain Risk Minimization with Pre-Trained Models