Maseru
Interpretable LLM-based Table Question Answering
Giang, null, Nguyen, null, Brugere, Ivan, Sharma, Shubham, Kariyappa, Sanjay, Nguyen, Anh Totti, Lecue, Freddy
Interpretability for Table Question Answering (Table QA) is critical, particularly in high-stakes industries like finance or healthcare. Although recent approaches using Large Language Models (LLMs) have significantly improved Table QA performance, their explanations for how the answers are generated are ambiguous. To fill this gap, we introduce Plan-of-SQLs ( or POS), an interpretable, effective, and efficient approach to Table QA that answers an input query solely with SQL executions. Through qualitative and quantitative evaluations with human and LLM judges, we show that POS is most preferred among explanation methods, helps human users understand model decision boundaries, and facilitates model success and error identification. Furthermore, when evaluated in standard benchmarks (TabFact, WikiTQ, and FetaQA), POS achieves competitive or superior accuracy compared to existing methods, while maintaining greater efficiency by requiring significantly fewer LLM calls and database queries.
- North America > United States > Iowa (0.07)
- North America > United States > Michigan (0.05)
- North America > United States > Tennessee (0.05)
- (39 more...)
- Workflow (1.00)
- Research Report > New Finding (0.46)
- Research Report > Experimental Study (0.46)
- Health & Medicine (1.00)
- Banking & Finance (1.00)
- Leisure & Entertainment > Sports > Tennis (0.46)
- Leisure & Entertainment > Sports > Golf (0.46)
Rulebreakers Challenge: Revealing a Blind Spot in Large Language Models' Reasoning with Formal Logic
Chan, Jason, Gaizauskas, Robert, Zhao, Zhixue
Formal logic has long been applied to natural language reasoning, but this approach can sometimes lead to conclusions that, while logically entailed, are factually inconsistent with the premises or are not typically inferred by humans. This study introduces the concept of "rulebreakers", which refers to instances where logical entailment diverges from factually acceptable inference. We present RULEBREAKERS, a novel dataset for evaluating Large Language Models' (LLMs) ability to distinguish between rulebreakers and non-rulebreakers. Focusing on modus tollens and disjunctive syllogism, we assess six state-of-the-art LLMs using RULEBREAKERS, measuring their performance in terms of token-level exact accuracy and model confidence. Our findings reveal that while most models perform poorly to moderately in recognizing rulebreakers, they demonstrate a latent ability to distinguish rulebreakers when assessed by their confidence levels. Further analysis suggests that the failure to recognize rulebreakers is potentially associated with the models' world knowledge and their attention distribution patterns. This research highlights the limitation of LLMs' reasoning capabilities, and contributes to the ongoing discussion on reasoning in LLMs.
ART: The Alternating Reading Task Corpus for Speech Entrainment and Imitation
Yuan, Zheng, de Jong, Dorina, Beňuš, Štefan, Nguyen, Noël, Feng, Ruitao, Sabo, Róbert, Fadiga, Luciano, D`Ausilio, Alessandro
We introduce the Alternating Reading Task (ART) Corpus, a collection of dyadic sentence reading for studying the entrainment and imitation behaviour in speech communication. The ART corpus features three experimental conditions - solo reading, alternating reading, and deliberate imitation - as well as three sub-corpora encompassing French-, Italian-, and Slovak-accented English. This design allows systematic investigation of speech entrainment in a controlled and less-spontaneous setting. Alongside detailed transcriptions, it includes English proficiency scores, demographics, and in-experiment questionnaires for probing linguistic, personal and interpersonal influences on entrainment. Our presentation covers its design, collection, annotation processes, initial analysis, and future research prospects.
- Africa > South Africa (0.04)
- Europe > Italy (0.04)
- Africa > Lesotho > Maseru > Maseru (0.04)
- (9 more...)
- Research Report > Experimental Study (0.93)
- Research Report > New Finding (0.93)
- Education (0.93)
- Leisure & Entertainment (0.68)
Flickr Africa: Examining Geo-Diversity in Large-Scale, Human-Centric Visual Data
Naggita, Keziah, LaChance, Julienne, Xiang, Alice
Biases in large-scale image datasets are known to influence the performance of computer vision models as a function of geographic context. To investigate the limitations of standard Internet data collection methods in low- and middle-income countries, we analyze human-centric image geo-diversity on a massive scale using geotagged Flickr images associated with each nation in Africa. We report the quantity and content of available data with comparisons to population-matched nations in Europe as well as the distribution of data according to fine-grained intra-national wealth estimates. Temporal analyses are performed at two-year intervals to expose emerging data trends. Furthermore, we present findings for an ``othering'' phenomenon as evidenced by a substantial number of images from Africa being taken by non-local photographers. The results of our study suggest that further work is required to capture image data representative of African people and their environments and, ultimately, to improve the applicability of computer vision models in a global context.
- Asia > Brunei (0.14)
- North America > Canada > Quebec > Montreal (0.06)
- Africa > Sierra Leone (0.06)
- (142 more...)
- Health & Medicine (0.92)
- Information Technology > Services (0.75)
- Government > Regional Government (0.46)
- Information Technology > Communications > Social Media (1.00)
- Information Technology > Artificial Intelligence > Vision (1.00)
How Disabled Students Benefit From Assistive Technology In Classrooms
Technology has modernized the system of education for students with various disabilities, making it easier for them to keep up with academic curriculums and even compete with their peers in classrooms. According to Open Colleges, most of the common disabilities can be categorized into any of the following classification -- Physical (students using wheelchairs, prosthetic limbs, or dealing with diseases such as muscular dystrophy, Lou Gehrig's disease, multiple sclerosis, etc), Sensory (students lacking in normal visual, hearing or speaking abilities), Cognitive (students with weaknesses when it comes to memory, self-expression, information processing, and other learning disabilities), Psychiatric (students may suffer from an array of challenges, ranging from social phobias, bipolar and/or other personality disorders), Health-related (students who have chronic illnesses like cancer, diabetes or epilepsy) A Palestinian child reads braille during a class at Al-Nour, which translates'we have seen,' Rehabilitation Center for the Visually Impaired, in Gaza City, Gaza Strip, May 7, 2006. Students who suffer from any form of disability might find it difficult to attend classes regularly, keep up with everything that is being taught and compete at the same level with children who are not plagued by the same impairments that they have. These students often need some extra assistance when it comes to performing academically. One of the best forms of assistance in today's times is the gift of technology.
- Asia > Middle East > Palestine > Gaza Strip > Gaza Governorate > Gaza (0.55)
- North America > United States > North Carolina (0.06)
- Europe > United Kingdom > England > Greater London > London (0.05)
- (2 more...)