Optimal Control of Logically Constrained Partially Observable and Multi-Agent Markov Decision Processes