Implementing a Weighted Majority Rule Ensemble Classifier

If you are interested in using the EnsembleClassifier, please note that it is now also available through scikit learn ( 0.17) as VotingClassifier. Here, I want to present a simple and conservative approach of implementing a weighted majority rule ensemble classifier in scikit-learn that yielded remarkably good results when I tried it in a kaggle competition. For me personally, kaggle competitions are just a nice way to try out and compare different approaches and ideas – basically an opportunity to learn in a controlled environment with nice datasets. Of course, there are other implementations of more sophisticated ensemble methods in scikit-learn, such as bagging classifiers, random forests, or the famous AdaBoost algorithm. However, as far as I am concerned, they all require the usage of a common "base classifier." In contrast, my motivation for the following approach was to combine conceptually different machine learning classifiers and use a majority vote rule.