[D] A2C/A3C: sharing the weights and same loss function? • r/MachineLearning

As Mnih paper explained (while the parameters θ of the policy and v of the value function are show to be separate, for generality we always share some of the parameters in practice with all non-output layers shared) it means that for the CNN part we share the parameters (weights). I understand that what you say is that the loss function is the sum of two variables calculated from two functions, the one calculated from policy loss and the other calculated from the value loss. And as consequence "Even though you add policy loss and the value loss at the end to create ONE loss function, they are effectively still two separate loss functions. "So when the backward pass flows through them, it is equally given to both functions without any interaction. " So as I understand you explain that there is still 2 gradients, one for the policy and one for the value because there is still 2 loss function.