Thought, the LLM can verbalize that the unknown city is Paris and use this fact to answer downstream questions.

Remarkably, the solutions learned for each individual task resemble those obtained by solving a kernel regression problem, revealing a novel connection between neural networks and kernel methods.