Throwing everything - including the kitchen sink - at a machine learning problem

It seems the more I read, the more confused I get - models, algorithms, surrogates; my head is spinning. Assume the dataset is in perfect condition - pure as the driven snow, no correlated features, no null in sight, nothing; and it has "enough" observations. To simplify, let's say we are looking at binary classification. Let's also say that we want to try four different algorithms: for example - logistic regression, naive Bayes, gradient boosted tree and multilayer perceptron. And, finally, let's assume that (since all this is for educational purposes), we have no issues with time, efficiency, computing power, computing budget and whatnot; we don't care if this is an overkill or if we're going after a fly with an elephant gun: we want to throw everything, including the kitchen sink, at the problem so we can extract every last ounce of performance when it's time to make predictions on totally unseen data.