Uni[MASK]: Unified Inference in Sequential Decision Problems