SMF-VO: Direct Ego-Motion Estimation via Sparse Motion Fields