Enhancing LLM-Based Human-Robot Interaction with Nuances for Diversity Awareness