The Hybrid Bootstrap: A Drop-in Replacement for Dropout

The field of machine learning offers many potent models for inference. Unfortunately, simply optimizing how well these models perform on a fixed training sample often leads to relatively poor performance on new test data compared to models that fit the training data less well. Regularization schemes are used to constrain the fitted model to improve performance on new data. One popular regularization tactic is to corrupt the training data with independently sampled noise. This constrains the model to work on data that is different from the original training data in a way that does not change the correct inference.