Goto

Collaborating Authors

 Maseru


Interpretable LLM-based Table Question Answering

Giang, null, Nguyen, null, Brugere, Ivan, Sharma, Shubham, Kariyappa, Sanjay, Nguyen, Anh Totti, Lecue, Freddy

arXiv.org Artificial Intelligence

Interpretability for Table Question Answering (Table QA) is critical, particularly in high-stakes industries like finance or healthcare. Although recent approaches using Large Language Models (LLMs) have significantly improved Table QA performance, their explanations for how the answers are generated are ambiguous. To fill this gap, we introduce Plan-of-SQLs ( or POS), an interpretable, effective, and efficient approach to Table QA that answers an input query solely with SQL executions. Through qualitative and quantitative evaluations with human and LLM judges, we show that POS is most preferred among explanation methods, helps human users understand model decision boundaries, and facilitates model success and error identification. Furthermore, when evaluated in standard benchmarks (TabFact, WikiTQ, and FetaQA), POS achieves competitive or superior accuracy compared to existing methods, while maintaining greater efficiency by requiring significantly fewer LLM calls and database queries.


Rulebreakers Challenge: Revealing a Blind Spot in Large Language Models' Reasoning with Formal Logic

Chan, Jason, Gaizauskas, Robert, Zhao, Zhixue

arXiv.org Artificial Intelligence

Formal logic has long been applied to natural language reasoning, but this approach can sometimes lead to conclusions that, while logically entailed, are factually inconsistent with the premises or are not typically inferred by humans. This study introduces the concept of "rulebreakers", which refers to instances where logical entailment diverges from factually acceptable inference. We present RULEBREAKERS, a novel dataset for evaluating Large Language Models' (LLMs) ability to distinguish between rulebreakers and non-rulebreakers. Focusing on modus tollens and disjunctive syllogism, we assess six state-of-the-art LLMs using RULEBREAKERS, measuring their performance in terms of token-level exact accuracy and model confidence. Our findings reveal that while most models perform poorly to moderately in recognizing rulebreakers, they demonstrate a latent ability to distinguish rulebreakers when assessed by their confidence levels. Further analysis suggests that the failure to recognize rulebreakers is potentially associated with the models' world knowledge and their attention distribution patterns. This research highlights the limitation of LLMs' reasoning capabilities, and contributes to the ongoing discussion on reasoning in LLMs.


ART: The Alternating Reading Task Corpus for Speech Entrainment and Imitation

Yuan, Zheng, de Jong, Dorina, Beňuš, Štefan, Nguyen, Noël, Feng, Ruitao, Sabo, Róbert, Fadiga, Luciano, D`Ausilio, Alessandro

arXiv.org Artificial Intelligence

We introduce the Alternating Reading Task (ART) Corpus, a collection of dyadic sentence reading for studying the entrainment and imitation behaviour in speech communication. The ART corpus features three experimental conditions - solo reading, alternating reading, and deliberate imitation - as well as three sub-corpora encompassing French-, Italian-, and Slovak-accented English. This design allows systematic investigation of speech entrainment in a controlled and less-spontaneous setting. Alongside detailed transcriptions, it includes English proficiency scores, demographics, and in-experiment questionnaires for probing linguistic, personal and interpersonal influences on entrainment. Our presentation covers its design, collection, annotation processes, initial analysis, and future research prospects.


Flickr Africa: Examining Geo-Diversity in Large-Scale, Human-Centric Visual Data

Naggita, Keziah, LaChance, Julienne, Xiang, Alice

arXiv.org Artificial Intelligence

Biases in large-scale image datasets are known to influence the performance of computer vision models as a function of geographic context. To investigate the limitations of standard Internet data collection methods in low- and middle-income countries, we analyze human-centric image geo-diversity on a massive scale using geotagged Flickr images associated with each nation in Africa. We report the quantity and content of available data with comparisons to population-matched nations in Europe as well as the distribution of data according to fine-grained intra-national wealth estimates. Temporal analyses are performed at two-year intervals to expose emerging data trends. Furthermore, we present findings for an ``othering'' phenomenon as evidenced by a substantial number of images from Africa being taken by non-local photographers. The results of our study suggest that further work is required to capture image data representative of African people and their environments and, ultimately, to improve the applicability of computer vision models in a global context.


How Disabled Students Benefit From Assistive Technology In Classrooms

International Business Times

Technology has modernized the system of education for students with various disabilities, making it easier for them to keep up with academic curriculums and even compete with their peers in classrooms. According to Open Colleges, most of the common disabilities can be categorized into any of the following classification -- Physical (students using wheelchairs, prosthetic limbs, or dealing with diseases such as muscular dystrophy, Lou Gehrig's disease, multiple sclerosis, etc), Sensory (students lacking in normal visual, hearing or speaking abilities), Cognitive (students with weaknesses when it comes to memory, self-expression, information processing, and other learning disabilities), Psychiatric (students may suffer from an array of challenges, ranging from social phobias, bipolar and/or other personality disorders), Health-related (students who have chronic illnesses like cancer, diabetes or epilepsy) A Palestinian child reads braille during a class at Al-Nour, which translates'we have seen,' Rehabilitation Center for the Visually Impaired, in Gaza City, Gaza Strip, May 7, 2006. Students who suffer from any form of disability might find it difficult to attend classes regularly, keep up with everything that is being taught and compete at the same level with children who are not plagued by the same impairments that they have. These students often need some extra assistance when it comes to performing academically. One of the best forms of assistance in today's times is the gift of technology.