Automatic essay grading (AEG) is a process in which machines assign a grade to an essay written in response to a topic, called the prompt. Zero-shot AEG is when we train a system to grade essays written to a new prompt which was not present in our training data. In this paper, we describe a solution to the problem of zero-shot automatic essay grading, using cognitive information, in the form of gaze behaviour. Our experiments show that using gaze behaviour helps in improving the performance of AEG systems, especially when we provide a new essay written in response to a new prompt for scoring, by an average of almost 5 percentage points of QWK.
What's the best way to prove you "know" something? A. Multiple choice tests B. Essays C. Interviews D. None of the above Go ahead: argue with the premise of the question. Oh yeah, you can't do that on multiple-choice tests. Essays can often better gauge what you know. Writing is integral to many jobs. But despite the fact that everyone can acknowledge that they're a more useful metric, we don't demand students write much on standardized tests because it's daunting to even imagine grading millions of essays.
In the year 1966 when computers still filled whole rooms, researcher Ellis Page at the University of Connecticut took the first steps towards automatic grading. Page was a true visionary of his generation. Computers was a relatively new thing a the thought of using them with text input rather than numbers must have seemed extremely novel to Page's peers. Besides, computers were mainly reserved for the most advanced tasks possible, and access to them was still highly restricted. Today however, the need for automated computer grading is soaring.
In this paper we promote introducing software verification and control flow graph similarity measurement in automated evaluation of students' programs. We present a new grading framework that merges results obtained by combination of these two approaches with results obtained by automated testing, leading to improved quality and precision of automated grading. These two approaches are also useful in providing a comprehensible feedback that can help students to improve the quality of their programs We also present our corresponding tools that are publicly available and open source. The tools are based on LLVM low-level intermediate code representation, so they could be applied to a number of programming languages. Experimental evaluation of the proposed grading framework is performed on a corpus of university students' programs written in programming language C. Results of the experiments show that automatically generated grades are highly correlated with manually determined grades suggesting that the presented tools can find real-world applications in studying and grading.
Via UT Dallas Mercury News Megan Zerez – For professors struggling to cope with stacks of papers to grade, new software -- developed by a UT Dallas researcher and powered by artificial intelligence -- may offer a long-term solution. Vincent Ng, a computer science professor who works with UT Dallas Human Language Technology Research Institute, is developing an automated grading system for longform essays. Ng said the goal of the technology is to remove the need for human graders altogether. "Essay grading is one of the very important applications of natural language processing," Ng said. "For one, it has a lot of commercial value.