Adapting Large Multimodal Models to Distribution Shifts: The Role of In-Context Learning