Teach CLIP to Develop a Number Sense for Ordinal Regression