A Simple Ensemble Strategy for LLM Inference: Towards More Stable Text Classification

In the field of marketing, accurate comprehension of customer's loyalty and preference is crucial as a customer relationship management (CRM) [1, 2]. In particular, as consumers increasingly post opinions on social media and review platform, user-generated contents (UGCs) has become essential resource for market research [3]. Textual data have been utilized for company's various decision-making, such as product evaluation, feature extraction, and recommendation systems [4, 5, 6, 7, 8]. To extract, utilize, and understand consumer preferences from textual data, pre-processing through assigning labels is essential; however, these are labor-intensive tasks for humans. Manual labelling such as Amazon Mechanical Turk (MTurk) is costly, while traditional natural language processing (NLP) methods require specialized skills. In addition, data quality of crowdsourcing remains a serious concern [9, 10]. Thus, handling big data becomes more challenging and often impractical despite the large amount of accumulated data. With the advance of large language models (LLMs), several studies have proposed automated annotation models using LLMs [11, 12, 13].