Rethinking Transformers in Solving POMDPs