Promoting Reflection and its Effect on Learning in a Programming Tutor

We studied the effect of post-practice reflection on learning, using programming tutors, and multiple-choice format for reflection. We conducted in-vivo controlled studies with introductory programming students from multiple schools over 3 semesters, and used mixed-factor ANOVA to analyze the collected data. We found that reflecting on the concept underlying each problem neither promotes greater learning, measured as pre-post increase in the average score per problem, nor promotes faster learning, measured as the problems solved per concept learned. We conjecture that the benefits of reflecting on the concept underlying each problem may be limited if a tutor already promotes deep understanding of the domain.