Goto

Collaborating Authors

 distinguish


Can NLP Models 'Identify', 'Distinguish', and 'Justify' Questions that Don't have a Definitive Answer?

arXiv.org Artificial Intelligence

Though state-of-the-art (SOTA) NLP systems have achieved remarkable performance on a variety of language understanding tasks, they primarily focus on questions that have a correct and a definitive answer. However, in real-world applications, users often ask questions that don't have a definitive answer. Incorrectly answering such questions certainly hampers a system's reliability and trustworthiness. Can SOTA models accurately identify such questions and provide a reasonable response? To investigate the above question, we introduce QnotA, a dataset consisting of five different categories of questions that don't have definitive answers. Furthermore, for each QnotA instance, we also provide a corresponding QA instance i.e. an alternate question that ''can be'' answered. With this data, we formulate three evaluation tasks that test a system's ability to 'identify', 'distinguish', and 'justify' QnotA questions. Through comprehensive experiments, we show that even SOTA models including GPT-3 and Flan T5 do not fare well on these tasks and lack considerably behind the human performance baseline. We conduct a thorough analysis which further leads to several interesting findings. Overall, we believe our work and findings will encourage and facilitate further research in this important area and help develop more robust models.


Articles

AI Magazine

WWTS (What Would Turing Say?) Turing's Imitation Game was a brilliant early proposed test of machine intelligence -- one that is still compelling today, despite the fact that in the hindsight of all that we've learned in the intervening 65 years we can see the flaws in his original test. And our field needs a good "Is it AI yet?" test more than ever today, with so many of us spending our research time looking under the "shallow processing of big data" lamppost. If Turing were alive today, what sort of test might he propose? If you are reading these words, surely you are already familiar with the Imitation Game proposed by Alan Turing (1950). Turing was heavily influenced by the World War II "game" of allied and axis pilots and ground stations each trying to fool the enemy into thinking they were friendlies.


artificial-intelligence-vs-machine-learning

#artificialintelligence

Neural networks use hyperparameters that distinguish objects and actions. A computer analyzes everything using numbers, so when it is fed an image, it sees that image as a set of numbers. If a neuron sees that an object's numbers fall within its range, then that neuron "fires." That means that if a neuron is trying to distinguish if a person is smiling, and the set of numbers it reads on an image falls within its hyperparameters, then the neuron would fire'yes,' and predict that the person is smiling.


fulltext

#artificialintelligence

A team of researchers has developed a mechanism that uses machine-learning algorithms to distinguish between genuine and counterfeit versions of the same product. "The underlying principle of our system stems from the idea that microscopic characteristics in a genuine product or a class of products -- corresponding to the same larger product line -- exhibit inherent similarities that can be used to distinguish these products from their corresponding counterfeit versions," says Subramanian, a professor at NYU's Courant Institute of Mathematical Sciences. Counterfeit goods represent a massive worldwide problem with nearly every high-valued physical object or product directly affected by this issue, the researchers note. The Entrupy method, by contrast, provides a non-intrusive solution to easily distinguish authentic versions of the product produced by the original manufacturer and fake versions of the product produced by counterfeiters.


an-advanced-ai-has-been-deployed-to-fight-against-hackers

#artificialintelligence

The grid connects computers in more than 40 countries from more than 170 research facilities, and works like a power grid to some extent, providing computing resources to facilities based on demand. This presents a unique cybersecurity challenge: keeping the massive globally-distributed grid secure while maintaining the computing power and storage unimpeded. Machine learning can train a system to detect potential threats while retaining the flexibility that it needs to provide computing power and storage on demand. If they work well protecting just the part of the grid that ALICE (A Large Ion Collider Experiment) uses, the team can deploy AI cybersecurity measures throughout the system.