ConvFill: Model Collaboration for Responsive Conversational Voice Agents