What I mean is training one layer at a time by freezing all but one and repeating this until the weights converge and give the best loss. I've seen narrow work on this in 2019 but it's surprising it hasn't been done before or at least explored. If there have been studies on this, what's the basic conclusion?

Yes, both hyperparameter tuning and architecture selection are optimization problems. Whether these are actually less difficult than NN training is debatable -- I think there are as around many papers on new architectures than there are on optimization techniques. Certainly, they are easier in the sense that a human can manually tune parameters and select an architecture which works reasonably well, but not select NN weights. Optimizing deep graphical models such as deep boltzmann machines is probably a more difficult optimization problem than training a neural network, depending on whether you consider DBMs a type of neural network.

Learning to Optimize is a recently proposed framework for learning optimization algorithms using reinforcement learning. In this paper, we explore learning an optimization algorithm for training shallow neural nets. Such high-dimensional stochastic optimization problems present interesting challenges for existing reinforcement learning algorithms. We develop an extension that is suited to learning optimization algorithms in this setting and demonstrate that the learned optimization algorithm consistently outperforms other known optimization algorithms even on unseen tasks and is robust to changes in stochasticity of gradients and the neural net architecture. More specifically, we show that an optimization algorithm trained with the proposed method on the problem of training a neural net on MNIST generalizes to the problems of training neural nets on the Toronto Faces Dataset, CIFAR-10 and CIFAR-100.