Goto

Collaborating Authors

 Media


Semantic Parsing with Candidate Expressions for Knowledge Base Question Answering

arXiv.org Artificial Intelligence

Semantic parsers convert natural language to logical forms, which can be evaluated on knowledge bases (KBs) to produce denotations. Recent semantic parsers have been developed with sequence-to-sequence (seq2seq) pre-trained language models (PLMs) or large language models, where the models treat logical forms as sequences of tokens. For syntactic and semantic validity, the semantic parsers use grammars that enable constrained decoding. However, the grammars lack the ability to utilize large information of KBs, although logical forms contain representations of KB elements, such as entities or relations. In this work, we propose a grammar augmented with candidate expressions for semantic parsing on a large KB with a seq2seq PLM. The grammar defines actions as production rules, and our semantic parser predicts actions during inference under the constraints by types and candidate expressions. We apply the grammar to knowledge base question answering, where the constraints by candidate expressions assist a semantic parser to generate valid KB elements. In experiments on two benchmarks, KQA Pro and Overnight, the constraints by candidate expressions increased the accuracy of our semantic parser, whether it was trained with strong supervision or weak supervision. Our semantic parser achieved state-of-the-art accuracies on KQA Pro and Overnight, and its implementation is publicly available at https://github.com/daehwannam/candexpr-sp.git.


Tesla's robotaxi event was long on Musk promises. Investors wanted more details.

The Japan Times

For a businessman who perpetually struggles with broken promises, Elon Musk gave himself quite a to-do list Thursday night at Tesla's long-awaited Hollywood unveiling of its driverless robotaxis. His slew of announcements during a 20-minute presentation were short on practical details, which pushed the stock to close nearly 9% lower at 217.80 on Friday. After traversing the fake streets of the Warner Bros movie studio set in a sleek, silver two-door "Cybercab" prototype, he promised that the company's popular Model 3 and Model Y vehicles would be able to operate without driver supervision in California and Texas by next year.


Yesterday's News: Benchmarking Multi-Dimensional Out-of-Distribution Generalisation of Misinformation Detection Models

arXiv.org Artificial Intelligence

This paper introduces misinfo-general, a benchmark dataset for evaluating misinformation models' ability to perform out-of-distribution generalisation. Misinformation changes rapidly, much quicker than moderators can annotate at scale, resulting in a shift between the training and inference data distributions. As a result, misinformation models need to be able to perform out-of-distribution generalisation, an understudied problem in existing datasets. We identify 6 axes of generalisation-time, event, topic, publisher, political bias, misinformation type-and design evaluation procedures for each. We also analyse some baseline models, highlighting how these fail important desiderata.


Honest AI: Fine-Tuning "Small" Language Models to Say "I Don't Know", and Reducing Hallucination in RAG

arXiv.org Artificial Intelligence

Hallucination is a key roadblock for applications of Large Language Models (LLMs), particularly for enterprise applications that are sensitive to information accuracy. To address this issue, two general approaches have been explored: Retrieval-Augmented Generation (RAG) to supply LLMs with updated information as context, and fine-tuning the LLMs with new information and desired output styles. In this paper, we propose Honest AI: a novel strategy to fine-tune "small" language models to say "I don't know" to reduce hallucination, along with several alternative RAG approaches. The solution ranked 1st in Task 2 for the false premise question. The alternative approaches include using RAG with search engine and knowledge graph results, fine-tuning base LLMs with new information and combinations of both approaches. Although all approaches improve the performance of the LLMs, RAG alone does not significantly improve the performance and fine-tuning is needed for better results. Finally, the hybrid approach achieved the highest score in the CRAG benchmark. In addition, our approach emphasizes the use of relatively small models with fewer than 10 billion parameters, promoting resource efficiency.


VERITAS-NLI : Validation and Extraction of Reliable Information Through Automated Scraping and Natural Language Inference

arXiv.org Artificial Intelligence

In today's day and age where information is rapidly spread through online platforms, the rise of fake news poses an alarming threat to the integrity of public discourse, societal trust, and reputed news sources. Classical machine learning and Transformer-based models have been extensively studied for the task of fake news detection, however they are hampered by their reliance on training data and are unable to generalize on unseen headlines. To address these challenges, we propose our novel solution, leveraging web-scraping techniques and Natural Language Inference (NLI) models to retrieve external knowledge necessary for verifying the accuracy of a headline. Our system is evaluated on a diverse self-curated evaluation dataset spanning over multiple news channels and broad domains. Our best performing pipeline achieves an accuracy of 84.3% surpassing the best classical Machine Learning model by 33.3% and Bidirectional Encoder Representations from Transformers (BERT) by 31.0% . This highlights the efficacy of combining dynamic web-scraping with Natural Language Inference to find support for a claimed headline in the corresponding externally retrieved knowledge for the task of fake news detection.


Automatic Speech Recognition with BERT and CTC Transformers: A Review

arXiv.org Artificial Intelligence

This review paper provides a comprehensive analysis of recent advances in automatic speech recognition (ASR) with bidirectional encoder representations from transformers BERT and connectionist temporal classification (CTC) transformers. The paper first introduces the fundamental concepts of ASR and discusses the challenges associated with it. It then explains the architecture of BERT and CTC transformers and their potential applications in ASR. The paper reviews several studies that have used these models for speech recognition tasks and discusses the results obtained. Additionally, the paper highlights the limitations of these models and outlines potential areas for further research. All in all, this review provides valuable insights for researchers and practitioners who are interested in ASR with BERT and CTC transformers.


On Goodhart's law, with an application to value alignment

arXiv.org Machine Learning

``When a measure becomes a target, it ceases to be a good measure'', this adage is known as {\it Goodhart's law}. In this paper, we investigate formally this law and prove that it critically depends on the tail distribution of the discrepancy between the true goal and the measure that is optimized. Discrepancies with long-tail distributions favor a Goodhart's law, that is, the optimization of the measure can have a counter-productive effect on the goal. We provide a formal setting to assess Goodhart's law by studying the asymptotic behavior of the correlation between the goal and the measure, as the measure is optimized. Moreover, we introduce a distinction between a {\it weak} Goodhart's law, when over-optimizing the metric is useless for the true goal, and a {\it strong} Goodhart's law, when over-optimizing the metric is harmful for the true goal. A distinction which we prove to depend on the tail distribution. We stress the implications of this result to large-scale decision making and policies that are (and have to be) based on metrics, and propose numerous research directions to better assess the safety of such policies in general, and to the particularly concerning case where these policies are automated with algorithms.


Ghost in the Shell's rad PS1 soundtrack is finally coming to the West

Engadget

The soundtrack to the spider-bot-crawling 1997 Ghost in the Shell game adaptation is coming to the West for the first time. Titled Ghost in the Shell: Megatech Body (as an ode to the Fuchikoma mech you pilot in the game), the soundtrack was produced by Takkyu Ishino. The PS1 game adaptation had late-90s gamers piloting a spider-like mech (first appearing in the 1991 manga), blasting enemies to smithereens with twin machine guns and guided missiles. Masamune Shirow, the original manga's author, wrote and illustrated its story and art design. But as 90s shooters often figured out, firing guns nonstop for hours on end is much better with a badass techno soundtrack pumping in the background like an energy drink for your ears. In addition to Ishino, it includes "warehouse-shaking bangers" from Mijk Van Dijk, The Advent, Joey Beltram and Brother from Another Planet (among others).


Engadget Podcast: Hunting data center vampires with Paris Marx

Engadget

What's that feature called on pixel phones? I forget what Android in general about Android specifics. But yes, there there was like a magic erase option there, too Yeah, I was going to say magic eraser, but that is a that's a clean thing it's something like that too, but It works really well like in terms of highlighting a specific object and removing it there are instances where it's too big and it can't like extrapolate like what should be a background so it looks really messy but sometimes like it just like smooths out a bright ugly object in the background was just like general unfocused stuff and that actually may be better.


Elon Musk showcases army of 30,000 'Optimus' robots designed to help with household chores including 'babysitting your kids' ... drawing comparisons to dystopian future depicted in I, Robot

Daily Mail - Science & tech

Elon Musk has showcased his army of 30,000 Tesla Optimus robots that are designed to help with household chores, prompting people to draw comparisons to the dystopian future depicted in I, Robot. In shocking and impressive footage, the humanoid robots were seen stiffly walking in single file across a stage while viewers stood jaw-dropped on the sidelines. Musk said attendees could walk up to the Optimus robots who would do things like serve drinks. 'At scale, you should be able to buy an Optimus robot for 20,000 to 30,000,' he said. 'It can walk your dog, mow your lawn, get the groceries, just be your friend.'