Induced Natural Language Rationales and Interleaved Markup Tokens Enable Extrapolation in Large Language Models