Machine Learning: Is exploring learning rate manually still necessary with an exponential decaying learning rate?

If we have an initial learning rate high enough and a suitable decay factor for exponentially decaying the learning rate over a certain number of epoch, is it still need for us to manually explore the learning rate? Because if all goes well I believe the learning rate can automatically be sampled over a huge range of epoch. However, if we start off with a less than optimal learning rate, assuming the loss does not diverge to infinity, would the loss be less optimal than we have started with the optimal learning rate, even if we could reach the optimal learning rate through decaying the initial learning rate over time? Does the answer differ for a convex/non-convex loss? Specifically for deep learning problems, is an exponential decaying learning rate able to sample the learning rate better than done manually?