Specifically, SegRefiner takes coarse masks as inputs and refines them using a discrete diffusion process.

In this paper, we explore a principal way to enhance the quality of object masks produced by different segmentation models. We propose a model-agnostic solution called SegRefiner, which offers a novel perspective on this problem by interpreting segmentation refinement as a data generation process. As a result, the refinement process can be smoothly implemented through a series of denoising diffusion steps. Specifically, SegRefiner takes coarse masks as inputs and refines them using a discrete diffusion process.

The overall workflow of the training and inference process are provided in Alg. 1 and Alg. 2. Model Architecture Following [ 9 ], we use a U-Net with 4-channel input and 1-channel output. Both input and output resolution is set to 256 256 . Training Settings All experiments are conducted on 8 NVIDIA RTX3090 GPUs with Pytorch. After a complete reverse diffusion process, the output is resized to the original size. We apply Non-Maximum Suppression (NMS, with 0.3 as threshold) to these patches to remove Our SegRefiner can robustly correct prediction errors both outside and inside the coarse mask.

Specifically, SegRefiner takes coarse masks as inputs and refines them using a discrete diffusion process.

In this paper, we explore a principal way to enhance the quality of object masks produced by different segmentation models. We propose a model-agnostic solution called SegRefiner, which offers a novel perspective on this problem by interpreting segmentation refinement as a data generation process. As a result, the refinement process can be smoothly implemented through a series of denoising diffusion steps. Specifically, SegRefiner takes coarse masks as inputs and refines them using a discrete diffusion process. To assess the effectiveness of SegRefiner, we conduct comprehensive experiments on various segmentation tasks, including semantic segmentation, instance segmentation, and dichotomous image segmentation.