Goto

Collaborating Authors

 Large Language Model


ScholarChemQA: Unveiling the Power of Language Models in Chemical Research Question Answering

arXiv.org Artificial Intelligence

Question Answering (QA) effectively evaluates language models' reasoning and knowledge depth. While QA datasets are plentiful in areas like general domain and biomedicine, academic chemistry is less explored. Chemical QA plays a crucial role in both education and research by effectively translating complex chemical information into readily understandable format. Addressing this gap, we introduce ScholarChemQA, a large-scale QA dataset constructed from chemical papers. This dataset reflects typical real-world challenges, including an imbalanced data distribution and a substantial amount of unlabeled data that can be potentially useful. Correspondingly, we introduce a QAMatch model, specifically designed to effectively answer chemical questions by fully leveraging our collected data. We first address the issue of imbalanced label distribution by re-weighting the instance-wise loss based on the inverse frequency of each class, ensuring minority classes are not dominated by majority ones during optimization. Next, we utilize the unlabeled data to enrich the learning process, generating a variety of augmentations based on a SoftMix operation and ensuring their predictions align with the same target, i.e., pseudo-labels. To ensure the quality of the pseudo-labels, we propose a calibration procedure aimed at closely aligning the pseudo-label estimates of individual samples with a desired ground truth distribution. Experiments show that our QAMatch significantly outperforms the recent similar-scale baselines and Large Language Models (LLMs) not only on our ScholarChemQA dataset but also on four benchmark datasets. We hope our benchmark and model can facilitate and promote more research on chemical QA.


FACTTRACK: Time-Aware World State Tracking in Story Outlines

arXiv.org Artificial Intelligence

While accurately detecting and correcting factual contradictions in language model outputs has become increasingly important as their capabilities improve, doing so is highly challenging. We propose a novel method, FACTTRACK, for tracking atomic facts and addressing factual contradictions. Crucially, FACTTRACK also maintains time-aware validity intervals for each fact, allowing for change over time. At a high level, FACTTRACK consists of a four-step pipeline to update a world state data structure for each new event: (1) decompose the event into directional atomic facts; (2) determine the validity interval of each atomic fact using the world state; (3) detect contradictions with existing facts in the world state; and finally (4) add new facts to the world state and update existing atomic facts. When we apply FACTTRACK to contradiction detection on structured story outlines, we find that FACTTRACK using LLaMA2-7B-Chat substantially outperforms a fair baseline using LLaMA2-7B-Chat, and achieves performance comparable to a GPT4 baseline. Moreover, when using GPT4, FACTTRACK significantly outperforms the GPT4 baseline.


A Perspectival Mirror of the Elephant

Communications of the ACM

Buddhism means different things to different cultures. To Westerners, Buddhism is generally associated with spirituality, meditation, and philosophy, while many Vietnamese associate it with the lunar calendar, holidays, mother god worship, and a lifestyle capable of bringing good luck. In Nepal, people typically see Buddhism as a protector that destroys bad karma. To move beyond these local views in an attempt to see the global picture, you might type "Buddhism" in Google's search bar. Instead of helping, however, the top 50 results skew strongly toward these distinct cultural impressions depending on the language you use for your query.


The AI PC revolution: 18 essential terms you need to know

PCWorld

These effects work in any application that uses your PC's built-in webcam, and it uses the NPU to apply these effects in a power-efficient way without draining your battery too quickly. While Windows Studio Effects are nice to have, I don't think they're reason enough to buy an AI PC, and certainly not a Meteor Lake-based AI PC. Intel's Lunar Lake is what the first AI PCs should have been. As more laptops ship with powerful NPUs, third-party application developers will likely start using them to add powerful AI features to their Windows desktop applications, putting the powerful PC hardware to use. I'm sure that's what Microsoft is hoping for, anyway. If you've gotten this far, congrats! You're now up to speed on all the most important AI PC terminologies, and you should understand enough now to see where all of this might be heading soon.


KWT-Tiny: RISC-V Accelerated, Embedded Keyword Spotting Transformer

arXiv.org Artificial Intelligence

University College Dublin, Ireland Abstract -- This paper explores the adaptation of Transformer - based models for edge devices through the quantis ation and hardware acceleration of the ARM Keyword Transformer (KWT) model on a RISC - V platform. The model was targeted to run on 64kB RAM in bare - metal C using a custom - developed edge AI library. KWT - 1 was retrained to be 369 times smaller, with only a 10 % loss in accuracy through reducing output classes from 35 to 2. The retraining and quantis ation reduced model size from 2.42 MB to 1.65 kB. The integration of custom RISC - V instructions that accelerated GELU and SoftMax operations enabled a 5x speedup and thus ~5x power reduction in inference, with inference clock cycle counts decreasing from 26 million to 5.5 million clock cycles while incurring a small area overhead of approximately 29 % . The results demonstrate a viable method for porting and accelerating Transformer - based models in low - power IoT devices.


SETTP: Style Extraction and Tunable Inference via Dual-level Transferable Prompt Learning

arXiv.org Artificial Intelligence

Text style transfer, an important research direction in natural language processing, aims to adapt the text to various preferences but often faces challenges with limited resources. In this work, we introduce a novel method termed S tyle E xtraction and T unable Inference via Dual-level T ransferable P rompt Learning (SETTP) for effective style transfer in low-resource scenarios. First, SETTP learns source style-level prompts containing fundamental style characteristics from high-resource style transfer. During training, the source style-level prompts are transferred through an attention module to derive a target style-level prompt for beneficial knowledge provision in low-resource style transfer. Additionally, we propose instance-level prompts obtained by clustering the target resources based on the semantic content to reduce semantic bias. We also propose an automated evaluation approach of style similarity based on alignment with human evaluations using ChatGPT -4. Our experiments across three resourceful styles show that SETTP requires only 1/20th of the data volume to achieve performance comparable to state-of-the-art (SOT A) methods. In tasks involving scarce data like writing style and role style, SETTP outperforms previous methods by 16.24%.


Impacts of Anthropomorphizing Large Language Models in Learning Environments

arXiv.org Artificial Intelligence

Similarly to the factors of anthropomorphism summarized by [11], we identified the following factors as relevant when Large Language Models (LLMs) are increasingly being used LLM-based chatbots are used in learning scenarios: The learning in learning environments to support teaching--be it as learning agent, i.e., chatbot, the learner itself, and environmental companions or as tutors [1]-[3]. With our contribution, we factors which influence the learner (see Figure 1). According to the media equation [4], people tend to respond to media in the same way as they would respond to another person. A study conducted by the Georgia Institute of Technology showed that chatbots can be successfully implemented in learning environments. As LLM-based chatbots such as OpenAI's GPT Looking at the agent, several factors can contribute to series are increasingly used in educational tools, it is important anthropomorphization. Cognitive intelligence refers to the to understand how the attribution processes to LLM-based ability to perceive, reason, and act on problems; to combine chatbots in terms of anthropomorphization affect learners' efficient, useful, goal-oriented, and autonomous actions with emotions.


Leveraging LLM Reasoning Enhances Personalized Recommender Systems

arXiv.org Artificial Intelligence

Recent advancements have showcased the potential of Large Language Models (LLMs) in executing reasoning tasks, particularly facilitated by Chain-of-Thought (CoT) prompting. While tasks like arithmetic reasoning involve clear, definitive answers and logical chains of thought, the application of LLM reasoning in recommendation systems (RecSys) presents a distinct challenge. RecSys tasks revolve around subjectivity and personalized preferences, an under-explored domain in utilizing LLMs' reasoning capabilities. Our study explores several aspects to better understand reasoning for RecSys and demonstrate how task quality improves by utilizing LLM reasoning in both zero-shot and finetuning settings. Additionally, we propose RecSAVER (Recommender Systems Automatic Verification and Evaluation of Reasoning) to automatically assess the quality of LLM reasoning responses without the requirement of curated gold references or human raters. We show that our framework aligns with real human judgment on the coherence and faithfulness of reasoning responses. Overall, our work shows that incorporating reasoning into RecSys can improve personalized tasks, paving the way for further advancements in recommender system methodologies.


Scaling CS1 Support with Compiler-Integrated Conversational AI

arXiv.org Artificial Intelligence

This paper introduces DCC Sidekick, a web-based conversational AI tool that enhances an existing LLM-powered C/C++ compiler by generating educational programming error explanations. The tool seamlessly combines code display, compile- and run-time error messages, and stack frame read-outs alongside an AI interface, leveraging compiler error context for improved explanations. We analyse usage data from a large Australian CS1 course, where 959 students engaged in 11,222 DCC Sidekick sessions, resulting in 17,982 error explanations over seven weeks. Notably, over 50% of interactions occurred outside business hours, underscoring the tool's value as an always-available resource. Our findings reveal strong adoption of AI-assisted debugging tools, demonstrating their scalability in supporting extensive CS1 courses. We provide implementation insights and recommendations for educators seeking to incorporate AI tools with appropriate pedagogical safeguards.


A Survey of AI Reliance

arXiv.org Artificial Intelligence

Artificial intelligence (AI) systems have become an indispensable component of modern technology. However, research on human behavioral responses is lagging behind, i.e., the research into human reliance on AI advice (AI reliance). Current shortcomings in the literature include the unclear influences on AI reliance, lack of external validity, conflicting approaches to measuring reliance, and disregard for a change in reliance over time. Promising avenues for future research include reliance on generative AI output and reliance in multi-user situations. In conclusion, we present a morphological box that serves as a guide for research on AI reliance.