Traditionally, each component of task-oriented dialogue systems is trained independently with different supervision.

Autonomous agents operating in real-world scenarios frequently encounter uncertainty and make decisions based on incomplete information.

What policy should be employed in a Markov decision process with uncertain parameters?

The solution in [5,4]requires aone-pass forward computation of marginal probabilities.

Neither (i) nor (ii) are known for general RMABs. Therefore, to capture the scheduling problems addressed inthiswork,weintroduce anewsubclass ofRMABs,Collapsing Bandits, distinguished by the following feature: when an arm is played, the agent fully observes its state, "collapsing" any uncertainty, but when an arm is passive, no observation is made and uncertainty evolves.