What are the other optimization problems in deep learning other than training neural nets?

Yes, both hyperparameter tuning and architecture selection are optimization problems. Whether these are actually less difficult than NN training is debatable -- I think there are as around many papers on new architectures than there are on optimization techniques. Certainly, they are easier in the sense that a human can manually tune parameters and select an architecture which works reasonably well, but not select NN weights. Optimizing deep graphical models such as deep boltzmann machines is probably a more difficult optimization problem than training a neural network, depending on whether you consider DBMs a type of neural network.