Genre
Text Box Size, Skill, and Iterative Practice in a Writing Task
Raine, Roxanne Benoit (University of Memphis) | Mintz, Lisa (University of Memphis) | Crossley, Scott A. (Georgia State University) | Dai, Jianmin (University of Memphis) | McNamara, Danielle S. (University of Memphis)
Although freewriting strategies are commonly taught in composition courses, there have been few empirical studies on freewriting. We address this gap by examining effects of prior writing skills (as measured by a pre-write essay), freewriting training, text-box size (1, 10, 20 lines), and repetitive writing on freewriting quality. Participants watched an agent-based vicarious learning freewriting instruction video or a control video including brief instructions on freewriting. After training, participants wrote six freewrites, two in each box size. Lesson delivery and text box size did not affect expert human ratings of the freewrites. Furthermore, participants did not benefit from writing successive freewrites regardless of their initial skill level. We describe how these results have been used to inform the design of Writing-Pal, an essay-writing intelligent tutoring system.
Automated Scenario Adaptation in Support of Intelligent Tutoring Systems
Niehaus, James Michael (Charles River Analytics, Inc.) | Li, Boyang (Georgia Institute of Technology) | Riedl, Mark (Georgia Institute of Technology)
Learners may develop expertise by experiencing numerous different but relevant situations. Computer games and virtual simulations can facilitate these training opportunities, however, because of the relative difficulty in authoring new scenarios, the increasing need for new and different scenarios becomes a bottleneck in the learning process. Furthermore, a one-size-fits-all scenario may not address all of the abilities, needs, or goals of a particular learner. To address these issues we present a novel technique, Automated Scenario Adaptation, to automatically “rewrite” narrative scenario content to suit individual learners’ needs and abilities and to incorporate recent changes from real world learning needs. Scenario adaptation acts as problem generation for intelligent tutoring systems, producing greater learning opportunities that facilitate engagement and continued learner involvement.
Predicting Changes in Level of Abstraction in Tutor Responses to Students
Lipschultz, Michael C. (University of Pittsburgh) | Litman, Diane J. (University of Pittsburgh) | Jordan, Pamela (University of Pittsburgh) | Katz, Sandra (University of Pittsburgh)
We examine a corpus of reflective tutorial dialogues between human tutor and student after the student completed introductory physics problems, to predict when the tutor abstracted from the student's preceding turn or when the tutor specialized from the student's preceding turn. Tutor abstraction occurs when the tutor repeats a segment of the student's turn using more general terms. Tutor specialization occurs when the tutor repeats a segment of the student's turn using more concrete terms. We find that features extracted from the reflective dialogue context produce the most predictive models. Also, the tutor abstracts more often when the student shows signs of working at a very detailed level for awhile, and prompts for specification when the student's responses are imprecise.
Motivational Impacts of a Game-Based Intelligent Tutoring System
Jackson, G. Tanner (University of Memphis) | McNamara, Danielle (University of Memphis)
iSTART is an intelligent tutoring system (ITS) designed to improve students’ reading comprehension. Previous studies have indicated that iSTART is successful; however, these studies have also indicated that students benefit most from long-term interactions that can become tedious and boring. A new game-based version of the system has been developed, called iSTART-ME (motivationally enhanced). Initial results from a usability study with iSTART-ME indicate that this system increases engagement and decreases boredom over time.
A Theoretical and Empirical Approach in Assessing Motivational Factors: From Serious Games To an ITS
Derbali, Lotfi (University of Montreal) | Chalfoun, Pierre (University of Montreal) | Frasson, Claude (University of Montreal)
This study investigates Serious Games (SG) to assess motivational factors appropriate to an Intelligent Tutoring System (ITS). An ITS can benefit from SG’ elements that can highly support learners’ motivation. Thus, identifying and assessing the effect that these factors may have on learners is a crucial step before attempting to integrate them into an ITS. We designed an experiment using a Serious Game and combined both the theoretical ARCS model of motivation and empirical physiological sensors (heart rate, skin conductance and EEG) to assess the effects of motivational factors on learners. We then identified physiological patterns correlated with one motivational factor in a Serious Game (Alarm triggers) associated with the Attention category of the ARCS model. The best result of three classifiers run on the physiological data has reached an accuracy of 73.8% in identifying learners’ attention level as being either above or below average. These results open the door to the possibility for an ITS to discriminate between attentive and inattentive learners.
Impact of Word Sense Disambiguation on Ordering Dictionary Definitions in Vocabulary Learning Tutors
Rosa, Kevin Dela (Carnegie Mellon University) | Eskenazi, Maxine (Carnegie Mellon University)
Past research has shown that dictionaries and glosses can be beneficial in computer assisted language learning, particularly in vocabulary learning. We propose that L2 vocabulary learners can benefit from the use of a dictionary whose definitions are sensitive to the provided reading context, and that advances in the natural language processing task of word sense disambiguation can be used to automatically order the definitions of such a dictionary. An in-vivo study was conducted with ESL students to investigate the effect that the order of definitions has on vocabulary learning using REAP, a computer based vocabulary tutor. Our results showed that students benefited from having the algorithmically determined best definitions listed at the top of the definition list. Furthermore, our results suggest that word sense disambiguation may currently be good enough for use in intelligent language tutoring environments.
Patterns of Word Usage in Expert Tutoring Sessions: Verbosity versus Quality
D' (University of Memphis) | Mello, Sidney
It is widely acknowledged that one-on-one human tutoring is one of the most effective ways to provide learning, however, the source of its effectiveness is still unclear. Tutor-centered, student-centered, and interaction hypotheses have been proposed as possible explanations of the effectiveness of human tutoring. Most research has addressed this question by analyzing tutorial sessions at the dialogue move or speech act level. The present paper adopts a different approach by focusing on word usage patterns in 50 naturalistic tutorial sessions between human students and expert tutors. Specifically, each unique word in the session was designated as a student initiative word, a tutor initiative word, or a shared-initiative word. Comparisons of the frequencies as well as the weights of the words assigned to each of these categories indicated that the student and tutor share initiative even though the tutor’s are considerably more verbose. The implications of the results for the development of an ITS that aspires to model expert tutors are discussed.
Exploring the Effects of Errors in Assessment and Time Requirements of Learning Objects in a Peer-Based Intelligent Tutoring System
Champaign, John (University of Waterloo) | Cohen, Robin (University of Waterloo)
We revisit a framework for designing peer-based intelligent tutoring systems motivated by McCalla's ecological approach, where learning is facilitated by the previous experiences of peers with a corpus of learning objects. Prior research demonstrated the value of a proposed algorithm for modeling student learning and for selecting the most beneficial learning objects to present to new students. In this paper, we first adjust the validation of this approach to demonstrate its ability to cope with errors in assessing the learning of student peers. We then deepen the representation of learning objects to reflect the expected time to completion and demonstrate how this may lead to more effective selection of learning objects for students, and thus more effective learning. As part of our exploration of these new adjustments, we offer insights into how the size of learning object repositories may affect student learning, suggesting future extensions for the model and its validation.
Learning a Tutorial Dialogue Policy for Delayed Feedback
Boyer, Kristy Elizabeth (North Carolina State University) | Phillips, Robert (North Carolina State University and Applied Research Associates, Inc.) | Ha, Eun Young (North Carolina State University) | Wallis, Michael (North Carolina State University and Applied Research Associates, Inc.) | Vouk, Mladen (North Carolina State University) | Lester, James (North Carolina State University)
Creating natural language tutorial dialogue systems that realize effective strategies is a central challenge for intelligent tutoring systems research. Traditional approaches generally require large development time, do not generalize well across domains, and do not match the flexibility and natural language sophistication of human tutors. A promising approach that may offer several benefits is data-driven system development, in which a dialogue policy is learned from corpora of human tutorial dialogue. To date these learning approaches typically focus on optimizing the tutor’s choice of act, and do not explicitly model the instances in which the tutor chose not to act. This paper reports on a hidden Markov modeling (HMM) approach within human textual tutorial dialogue that explicitly represents the tutors’ choices not to intervene. The results show that an HMM that models tutor non-interventions predicts tutor moves significantly better than a model that does not explicitly represent the non-interventions. The findings have implications for automatically modeling tutorial strategies and for learning dialogue policies from corpora.
Robustness of Filter-Based Feature Ranking: A Case Study
Altidor, Wilker (Florida Atlantic University) | Khoshgoftaar, Taghi M. (Florida Atlantic University) | Hulse, Jason Van (Florida Atlantic University)
The filter model of feature selection has been well studied. In previous studies, classification performance has traditionally been proposed as a way to evaluate filter solutions. In this study, a new method of comparing feature ranking techniques is presented providing a straightforward approach for quantifying individual filters’ robustness to class noise. Six commonly-used filters, plus one which is rarely used, are investigated regarding their ability to retain, in the presence of class noise, strong classification performance. Three classifiers and one classification performance metric are considered. The experimental results of this study show that Gain Ratio, one of the well known and widely used filters, is very sensitive to class noise. ReliefF offers the best results with both the NB and kNN learners while Signal-to-noise, though not as widely used in the literature as the others, outperforms all the filters with the SVM learner.