Bot Meets Shortcut: How Can LLMs Aid in Handling Unknown Invariance OOD Scenarios?