Zipper: Addressing Degeneracy in Algorithm-Agnostic Inference