Markov Models
Automated Dynamic Mechanism Design
For environments with large time horizons, we show that the principal's optimal
Universal Off-Policy Evaluation
Finally, we also discuss UnO's applicability in various settings, including fully observable, partially observable (i.e.,
Appendix A A Stochastic Markov Model of a 2 Server Load Balancing Problem
Similar to the proof of Proposition 12, given the stability constraint in Eq. Eq. (4), we have C 0, l Theorem 14. Multi-agent load balancing is MPG with the VBF Solid and dashed arrows represent deterministic and non-deterministic procedures respectively. Real-world network applications can be CPU-bound or IO-bound [47, 48]. The simulator allows configuring applications that require multi-stage processes switching between CPU/IO queues (Figure 1b). Two different processing models are used for CPU and IO queues, respectively.
Robust Anytime Learning of Markov Decision Processes
Markov decision processes (MDPs) are formal models commonly used in sequential decision-making.
Continuously-Tempered PDMP samplers
However, these methods can struggle to sample from multi-modal or heavy-tailed distributions.
MDNS: Masked Diffusion Neural Sampler via Stochastic Optimal Control
Zhu, Yuchen, Guo, Wei, Choi, Jaemoo, Liu, Guan-Horng, Chen, Yongxin, Tao, Molei
We study the problem of learning a neural sampler to generate samples from discrete state spaces where the target probability mass function $π\propto\mathrm{e}^{-U}$ is known up to a normalizing constant, which is an important task in fields such as statistical physics, machine learning, combinatorial optimization, etc. To better address this challenging task when the state space has a large cardinality and the distribution is multi-modal, we propose $\textbf{M}$asked $\textbf{D}$iffusion $\textbf{N}$eural $\textbf{S}$ampler ($\textbf{MDNS}$), a novel framework for training discrete neural samplers by aligning two path measures through a family of learning objectives, theoretically grounded in the stochastic optimal control of the continuous-time Markov chains. We validate the efficiency and scalability of MDNS through extensive experiments on various distributions with distinct statistical properties, where MDNS learns to accurately sample from the target distributions despite the extremely high problem dimensions and outperforms other learning-based baselines by a large margin. A comprehensive study of ablations and extensions is also provided to demonstrate the efficacy and potential of the proposed framework.