Due to its iconic nature ( i.e ., perceptual resemblance to or natural association with the referent), drawings serve as a powerful tool to communicate concepts transcending language barriers (Fay et al., 2014). In fact, we humans started to use drawings to convey messages dating back to 40,000-60,000 years ago (Hoffmann et al., 2018; Hawkins et al., 2019).

Combining these inequalities, along with the fact γ < 1, we conclude that g (s) 0 for all s S . Hence, as γ < 1, we conclude that g (s) 0 for all s S . Given we know both agents' policies ( As R is closed under addition, we see that c = λa + µb R . Therefore, all states can be accessed from s. If the starting state is ephemeral, it is clear that we can add a constant to its rewards independently of all other states' rewards, as this will not affect decision Proof of Theorem 4. We first prove the sufficiency statement.

Gaussian process with a customized kernel function and an interpretable predictor.

Multi-agent control is a central theme in the Cyber-Physical Systems (CPS) . However, current control methods either receive non-Markovian states due to insufficient sensing and decentralized design, or suffer from poor convergence.

We introduce a constrained optimization method for policy gradient reinforcement learning, which uses a virtual trust region to regulate each policy update. In addition to using the proximity of one single old policy as the normal trust region, we propose forming a second trust region through another virtual policy representing a wide range of past policies. We then enforce the new policy to stay closer to the virtual policy, which is beneficial if the old policy performs poorly. More importantly, we propose a mechanism to automatically build the virtual policy from a memory of past policies, providing a new capability for dynamically learning appropriate virtual trust regions during the optimization process. Our proposed method, dubbed Memory-Constrained Policy Optimization (MCPO), is examined in diverse environments, including robotic locomotion control, navigation with sparse rewards and Atari games, consistently demonstrating competitive performance against recent on-policy constrained policy gradient methods.