Social robots can be used to tutor children in one-on-one interactions. Because students have different learning needs, they consequently require complex, non-scripted teaching behaviors that adapt to the learning needs of each child. As a result of this, robot tutors are more effective given a means of adaptively customizing the pace and content of a student's curriculum. In this paper we propose a reinforcement learning-based approach that affords such capabilities to a tutoring robot, with the goals of fostering measurable learning gains and sustained engagement. We outline an architecture in which the robot uses reinforcement learning to adapt the difficulty of its exercises. Further, we describe a proposed study capable of evaluating the effectiveness of our Intelligent Tutoring System.