Break the Chain: Large Language Models Can be Shortcut Reasoners