CoTEVer: Chain of Thought Prompting Annotation Toolkit for Explanation Verification

Chain-of-thought (CoT) prompting enables large language models (LLMs) to solve complex reasoning tasks by generating an explanation before the final prediction. Despite it's promising ability, a critical downside of CoT prompting is that the performance is greatly affected by the factuality of the generated explanation. To improve the correctness of the explanations, fine-tuning language models with explanation data is needed. However, there exists only a few datasets that can be used for such approaches, and no data collection tool for building them. Thus, we introduce CoTEVer, a tool-kit for annotating the factual correctness of generated explanations and collecting revision data of wrong explanations. Figure 1: Example of Explanation Verification and Answer Furthermore, we suggest several use cases Verification of GPT-3's output. Explanation Verification where the data collected with CoTEVer can requires additional knowledge which makes it be utilized for enhancing the faithfulness of hard for annotators to intuitively write a revised explanation explanations. Our toolkit is publicly available and answer.