PrefMMT: Modeling Human Preferences in Preference-based Reinforcement Learning with Multimodal Transformers