CBEval: A framework for evaluating and interpreting cognitive biases in LLMs

Large language models have emerged as powerful instruments that can automate reasoning process in a host of domains ranging from Olympiad level math problem solving [1] to code generation [2] and financial planning [3]. LLMs have seen widespread adoption among people to automate and refine tasks involving decision making, logical thinking and critical reasoning. However, despite such promising results on numerous benchmarks, LLMs still possess surprising knowledge gaps caused due to a host of reasons varying from engineering heuristics such as tokenization [4] to limitations in the training data itself, such as data bias or a lack of up-to-date information. Furthermore, due to being trained on extensive data accumulated and refined by humans over the years, these LLMs present grounds for inherent cognitive bias as characterized in humans [5]. While there have been promising demonstrations of GPT-based models on tasks involving creativity [6] and critical thinking [7], slight variations in input prompt requests can lead to vastly different output responses[8]. Furthermore, due to being next token predictors scaled to a huge text, the world models formed by language models possess knowledge gaps unapparent on the surface level.