However, FACMAClearnsacentralised butfactored critic,which combines per-agent utilities into the joint action-value function via a non-linear monotonic function, as inQMIX, apopular multi-agentQ-learning algorithm. However,unlikeQMIX, there are no inherent constraints on factoring the critic. We thus also employ a nonmonotonic factorisation and empirically demonstrate that its increased representational capacity allows it to solve some tasks that cannot be solved with monolithic, ormonotonically factored critics.

Recurrent experience replay in distributed reinforcement learning.

Moreover, we extend ICQ to multi-agent tasks by decomposing the joint-policy under the implicit constraint. Experimental results demonstrate that the extrapolation error is successfully controlled within a reasonable range and insensitive to the number of agents.

Multi-agent games allow sophisticated interactions between agents and environment. Feasible solutions may require non-trivial intra-agent coordination, which leads to substantially more complex strategies than the single-agent setting.

Onenaiveapproach to incorporate this ability into AI is through future trajectory prediction using model-based RL [40,19,31,54,28,26].