CS 540 Lecture Notes: Machine Learning

The C5.0 algorithm uses the Max-Gain method of selecting the best attribute. H measures the information content or entropy in bits (i.e., number of yes/no questions that must be asked) associated with a set S of examples, which consists of the subset P of positive examples and subset N of negative examples. Note: 0 H(P,N) 1, where 0 no information, and 1 maximum information. Half the examples in S are positive and half are negative. Say all of the examples in S are positive and none are negative.