Rule by Rule: Learning with Confidence through Vocabulary Expansion