Selective Matching Losses -- Not All Scores Are Created Equal