Learning Compact Neural Networks with Deep Overparameterised Multitask Learning

The left and right singular vectors are trained with all task losses, and the diagonal matrices are trained using taskspecific Compact neural network offers many benefits for losses. Our design is mainly inspired by analytical real-world applications. However, it is usually studies on overparameterised networks for MTL [Lampinen challenging to train the compact neural networks and Ganguli, 2018] that the training/test error dynamics depends with small parameter sizes and low computational on the time-evolving alignment of the network parameters costs to achieve the same or better model performance to the singular vectors of the training data, and a quantifiable compared to more complex and powerful task alignment describing the transfer benefits among architecture. This is particularly true for multitask multiple tasks depends on the singular values and input feature learning, with different tasks competing for resources.