Using Randomness Effectively in Deep Learning

I believe the performance benefits, ability to generalize, and robustness are all too good to ignore. Thus, I've written/talked a lot about it throughout my content. Recently, a reader of mine reached out with an interesting question. He wanted to know why it was that randomness in aspects such as Data Augmentation, but not in selecting features (Garbage In, Garbage Out). I figured this would make for a good topic since I stress the integration of noise and randomness into machine learning pipelines, but haven't covered why it works so well.