What Makes a Reward Model a Good Teacher? An Optimization Perspective