Equilibrium Discovery in Modular Deep Learning Architectures

Deep Learning (DL) systems will evolve from the current monolithic systems, to more modular systems. The traditional DL system is trained end-to-end with a single objective function and optimization algorithm. We however are already seeing newer systems like GANs, that involve more than one DL system. GANs employ a generator and a discriminator, that are in an adversarial relationship, competing against each other. The main difficulty of training GANs is that finding an equilibrium is difficult.