A Minimax-MDP Framework with Future-imposed Conditions for Learning-augmented Problems