Collaborating Authors

The End Of Multiple Choice? The Quest To Create Accurate Robot Essay Graders

AITopics Original Links

What's the best way to prove you "know" something? A. Multiple choice tests B. Essays C. Interviews D. None of the above Go ahead: argue with the premise of the question. Oh yeah, you can't do that on multiple-choice tests. Essays can often better gauge what you know. Writing is integral to many jobs. But despite the fact that everyone can acknowledge that they're a more useful metric, we don't demand students write much on standardized tests because it's daunting to even imagine grading millions of essays.

How fast can YOU find the dog's hidden master?

Daily Mail - Science & tech

There is a new optical illusion sweeping the web - and it could well be the toughest yet. Hidden in the vintage illustration of a dog is the head of his master - but can you spot him? The image, posted by Playbuzz, dates back to the turn of the century and was the face of a trade card used as an early advertising gimmick. How fast can you find the dog's master in this vintage illustration from the turn of the century? If you look closely, you can spot the dog's owner in the middle of the picture - with Spot's ear acting as his hat.

Crowdsourcing Multiple Choice Science Questions Machine Learning

We present a novel method for obtaining high-quality, domain-targeted multiple choice questions from crowd workers. Generating these questions can be difficult without trading away originality, relevance or diversity in the answer options. Our method addresses these problems by leveraging a large corpus of domain-specific text and a small set of existing questions. It produces model suggestions for document selection and answer distractor choice which aid the human question generation process. With this method we have assembled SciQ, a dataset of 13.7K multiple choice science exam questions (Dataset available at We demonstrate that the method produces in-domain questions by providing an analysis of this new dataset and by showing that humans cannot distinguish the crowdsourced questions from original questions. When using SciQ as additional training data to existing questions, we observe accuracy improvements on real science exams.

Data Mining Patterns of Thought

AAAI Conferences

Modern educational and psychological measurements are governed by models that do not allow for identification of patterns of student thought. However, in many situations, including diagnostic assessment, it is more important to understand student thought than to score it. We propose using entropy-based clustering to group responses to both a standard achievement test and a test specifically designed to reveal different facets of student thinking. We show that this approach is able to identify patterns of thought in these domains, although there are limitations to what information can be obtained from multiple choice responses alone. .

Tricky puzzle claims just one in every 2,000 people can solve it

Daily Mail - Science & tech

There is a new numbers game taking hold of the Internet - and it's a very tricky one. The latest offering from Playbuzz claims that just one person in every 2,000 will be able to pass the puzzle. The aim of the game is to try to work out what shape or letter the number 2's in each image form. Can you pass the latest test by Playbuzz? See if you can find the 2's and what shape they are It might sound easy enough but be warned, the number 2's are hidden among lots more number 5's.