Appendix to Weakly Coupled Deep Q-Networks

We prove part the first part of the proposition (weak duality) by induction. It is well-known that, by the value iteration algorithm's convergence, Q Consider a state s S and a feasible action a A(s). We use an induction proof. This can be established by shifting the origin of the coordinate system. We use the following lemma from [6] to bound the accumulated noise.