A Direct Approach for Handling Contextual Bandits with Latent State Dynamics