Robust Preference Alignment via Directional Neighborhood Consensus