CogSketch

AAAI Conferences

CogSketch is an open-domain sketch understanding system. CogSketch takes a unique approach to sketch understanding that focuses less on low-level recognition and more on highlevel reasoning with sketches.


Sketch Worksheets: A Sketch-Based Educational Software System

AAAI Conferences

Intelligent tutoring systems and learning environments can provide important benefits for education, but few have been developed for heavily spatial domains. One bottleneck has been the lack of rich models of visual and conceptual processing in sketch understanding, so that what students draw can be interpreted in a human-like way. This paper describes Sketch Worksheets, a form of sketch-based educational software that mimics aspects of pencil and paper worksheets commonly found in classrooms, but provides on-the-spot feedback and support for richer off-line assessments. The basic architecture of sketch worksheets is described, including an authoring environment that allows non-developers to create them and a coach that uses analogy to compare student and instructor sketches as a means to provide feedback. A pilot experiment where sketch worksheets were used successfully in a college geoscience class in Fall 2009 is summarized to show the potential of the idea.


Sketch Worksheets in STEM Classrooms: Two Deployments

AAAI Conferences

Sketching can be a valuable tool for science education, but it is currently underutilized. Sketch worksheets were developed to help change this, by using AI technology to give students immediate feedback and to give instructors assistance in grading. Sketch worksheets use visual representations automatically computed by CogSketch, which are combined with conceptual information from the OpenCyc ontology. Feedback is provided to students by comparing an instructor’s sketch to a student’s sketch, using the Structure-Mapping Engine. This paper describes our experiences in deploying sketch worksheets in two types of classes: Geoscience and AI. Sketch worksheets for introductory geoscience classes were developed by geoscientists at University of Wisconsin-Madison, authored using CogSketch and used in classes at both Wisconsin and Northwestern University. Sketch worksheets were also developed and deployed for a knowledge representation and reasoning course at Northwestern. Our experience indicates that sketch worksheets can provide helpful on-the-spot feedback to students, and significantly improve grading efficiency, to the point where sketching assignments can be more practical to use broadly in STEM education.


Solving Everyday Physical Reasoning Problems by Analogy using Sketches

AAAI Conferences

Understanding common sense reasoning about the physical world is one of the goals of qualitative reasoning research. This paper describes how we combine qualitative mechanics and analogy to solve everyday physical reasoning problems posed as sketches. The problems are drawn from the Bennett Mechanical Comprehension Test, which is used to evaluate technician candidates. We discuss sketch annotations, which define conceptual quantities in terms of visual measurements, how modeling decisions are made by analogy, and how analogy can be used to frame comparative analysis problems. Experimental results support the plausibility of this approach.


Using Quantitative Information to Improve Analogical Matching Between Sketches

AAAI Conferences

Qualitative representations are suitable for sketch understanding systems because they highlight important relationships while leaving out details that are not essential for conceptual understanding. These representations can be used to perform spatial analogies between sketches, which determine qualitative similarities and differences. However, there are cases where including quantitative information is necessary for accurately representing a sketch. We describe a method for using quantitative information to constrain qualitative spatial analogies. The utility of this method is demonstrated in the context of a sketch-based educational software system. Importantly, using quantitative information to improve analogical matches is not domain-specific. It can be used in any situation where qualitative and quantitative spatial information must be combined to accurately interpret a sketch. This approach has the potential to improve sketch understanding in educational software applications for highly spatial domains.