Pursuing Minimal Sufficiency in Spatial Reasoning