SteerVLM: Robust Model Control through Lightweight Activation Steering for Vision Language Models