A Language Model's Guide Through Latent Space