The CUTE benchmark showed that LLMs struggle with character understanding in English. We extend it to more languages with diverse scripts and writing systems, introducing EXECUTE. Our simplified framework allows easy expansion to any language. Tests across multiple LLMs reveal that challenges in other languages are not always on the character level as in English. Some languages show word-level processing issues, some show no issues at all. We also examine sub-character tasks in Chinese, Japanese, and Korean to assess LLMs' understanding of character components.

This study investigates the influence of Japanese honorifics on the responses of large language models (LLMs) when explaining the law of conservation of momentum. We analyzed the outputs of six state-of-the-art AI models, including variations of ChatGPT, Coral, and Gemini, using 14 different honorific forms. Our findings reveal that honorifics significantly affect the quality, consistency, and formality of AI-generated responses, demonstrating LLMs' ability to interpret and adapt to social context cues embedded in language. Notable variations were observed across different models, with some emphasizing historical context and derivations, while others focused on intuitive explanations. The study highlights the potential for using honorifics to adjust the depth and complexity of AI-generated explanations in educational contexts. Furthermore, the responsiveness of AI models to cultural linguistic elements underscores the importance of considering cultural factors in AI development for educational applications. These results open new avenues for research in AI-assisted education and cultural adaptation in AI systems, with significant implications for personalizing learning experiences and developing culturally sensitive AI tools for global education.