Understanding Prior Bias and Choice Paralysis in Transformer-based Language Representation Models through Four Experimental Probes