ALERT: Adapting Language Models to Reasoning Tasks