PuMer: Pruning and Merging Tokens for Efficient Vision Language Models