Distilling Functional Rearrangement Priors from Large Models