Value Function Decompositionfor Iterative Designof Reinforcement Learning Agents

In BW, an include: areforwardprogress, failur ), acostcontr ), ashapingrehead). Require:Experience B; twinQ-function 1, 2 (with parameters 1, 2; policyparameter ; discount ; entrop ; learningrates q, ; targetnetw ; Boolean 1: Sampletransition(s, a, r,0) B.r2Rm is 2: Samplepolica0 ( |s0; )andu ( |s; ) 3: rm+1 log (a0|s0; ).Extend 4: j argmin