Less is More: Efficient Model Merging with Binary Task Switch