Latent Factor Models Meets Instructions:Goal-conditioned Latent Factor Discovery without Task Supervision