CALI: An Online Scribble Recognizer for Calligraphic Interfaces

AAAI Conferences

CALI is a fast, simple and compact online recognizer that identifies Scribbles (multi-stroke geometric shapes) drawn with a stylus on a digitizing tablet. Our method is able to identify shapes of different sizes and rotated at arbitrary angles, drawn with dashed, continuous strokes or overlapping lines. We use temporal adjacency to allow users to input the most common shapes in drawing such as triangles, lines, rectangles, circles, diamonds and ellipses, using multiple strokes. We have further extended this approach to identify useful shapes such as arrows, crossing lines and unistroke gesture commands and have developed a library of software components to make this software generally available. The recognition algorithm uses Fuzzy Logic and geometric features, combined with an extensible set of heuristics to classify scribbles. More recently we developed a trainable version of the recognizer to allow users to easily add new shape classes to the initial core set. Evaluation results show recognition rates over 97% for the non-trainable and 95% for the trainable version.


Grouping Strokes into Shapes in Hand-Drawn Diagrams

AAAI Conferences

Objects in freely-drawn sketches often have no spatial or temporal separation, making object recognition difficult. We present a two-step stroke-grouping algorithm that first classifies individual strokes according to the type of object to which they belong, then groups strokes with like classifications into clusters representing individual objects. The first step facilitates clustering by naturally separating the strokes, and both steps fluidly integrate spatial and temporal information. Our approach to grouping is unique in its formulation as an efficient classification task rather than, for example, an expensive search task. Our single-stroke classifier performs at least as well as existing single-stroke classifiers on text vs. nontext classification, and we present the first three-way single-stroke classification results. Our stroke grouping results are the first reported of their kind; our grouping algorithm correctly groups between 86% and 91% of the ink in diagrams from two domains, with between 69% and 79% of shapes being perfectly clustered.


Mechanix: A Sketch-Based Tutoring and Grading System for Free-Body Diagrams

AI Magazine

In this article, we introduce Mechanix, a sketch-based deployed tutoring system for engineering students enrolled in statics courses. Our system not only allows students to enter planar truss and free-body diagrams into the system, just as they would with pencil and paper, but our system also checks the student's work against a hand-drawn answer entered by the instructor, and then returns immediate and detailed feedback to the student. Students are allowed to correct any errors in their work and resubmit until the entire content is correct and thus all of the objectives are learned. Since Mechanix facilitates the grading and feedback processes, instructors are now able to assign more free-response questions, increasing teacher's knowledge of student comprehension. Furthermore, the iterative correction process allows students to learn during a test, rather than simply display memorized information.


Mechanix: A Sketch-Based Tutoring and Grading System for Free-Body Diagrams

AI Magazine

Introductory engineering courses within large universities often have annual enrollments which can reach up to a thousand students. It is very challenging to achieve differentiated instruction in classrooms with class sizes and student diversity of such great magnitude. Professors can only assess whether students have mastered a concept by using multiple choice questions, while detailed homework assignments, such as planar truss diagrams, are rarely assigned because professors and teaching assistants would be too overburdened with grading to return assignments with valuable feedback in a timely manner. In this paper, we introduce Mechanix, a sketch-based deployed tutoring system for engineering students enrolled in statics courses. Our system not only allows students to enter planar truss and free body diagrams into the system just as they would with pencil and paper, but our system checks the student's work against a hand-drawn answer entered by the instructor, and then returns immediate and detailed feedback to the student. Students are allowed to correct any errors in their work and resubmit until the entire content is correct and thus all of the objectives are learned. Since Mechanix facilitates the grading and feedback processes, instructors are now able to assign free response questions, increasing teacher's knowledge of student comprehension. Furthermore, the iterative correction process allows students to learn during a test, rather than simply displaying memorized information.


Combining Geometry and Domain Knowledge to Interpret Hand-Drawn Diagrams

AAAI Conferences

We present a sketch understanding system for networklike diagrams consisting of symbols linked together. This system employs a novel parser to automatically extract symbols from a continuous stream of pen strokes. The parser uses geometric information to enumerate candidate symbols, and then uses domain knowledge to prune away unlikely candidates. The candidates are classified with a novel, domainindependent, probabilistic, feature-based symbol recognizer. Domain knowledge and context are used to correct parsing and recognition errors. To demonstrate our system, we used it to create a sketch-based interface for an electric circuit analysis program.