Despite its simplicity, guaranteeing efficiency andwhether sampling theactions fromtheposterior could leadtounbounded future trajectories is unknown.

Large language models (LLMs) have achieved a remarkable performance on diverse tasks across multiple domains, as reported in recent surveys [Wei et al., 2022, Bubeck et al., 2023].