One Network Fits All? Modular versus Monolithic Task Formulations in Neural Networks