Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition