SAPA: Similarity-Aware Point Affiliation for Feature Upsampling