CultureLLM: Incorporating Cultural Differences into Large Language Models

Large language models (LLMs) have been observed to exhibit bias towards certain cultures due to the predominance of training data obtained from English corpora. Considering that multilingual cultural data is often expensive to procure, existing methodologies address this challenge through prompt engineering or culture-specific pre-training. In this paper, we propose CultureLLM, a cost-effective solution to integrate cultural differences into LLMs. CultureLLM employs the World Value Survey (WVS) as seed data and generates semantically equivalent training data through the proposed semantic data augmentation. Extensive experiments conducted on 60 culture-related datasets reveal that CultureLLM significantly surpasses various counterparts such as GPT-3.5 (by 8.1 \%) and Gemini Pro (by 9.5 \%), demonstrating performance comparable to or exceeding that of GPT-4.