When Does Confidence-Based Cascade Deferral Suffice?