Partially Rewriting a Transformer in Natural Language