Goto

Collaborating Authors

 canberra


Hybrid Deep Learning Model to Estimate Cognitive Effort from fNIRS Signals

Sharmin, Shayla, Barmaki, Roghayeh Leila

arXiv.org Artificial Intelligence

This study estimates cognitive effort based on functional near-infrared spectroscopy data and performance scores using a hybrid DeepNet model. The estimation of cognitive effort enables educators to modify material to enhance learning effectiveness and student engagement. In this study, we collected oxygenated hemoglobin using functional near-infrared spectroscopy during an educational quiz game. Participants (n=16) responded to 16 questions in a Unity-based educational game, each within a 30-second response time limit. We used DeepNet models to predict the performance score from the oxygenated hemoglobin, and compared traditional machine learning and DeepNet models to determine which approach provides better accuracy in predicting performance scores. The result shows that the proposed CNN-GRU gives better performance with 73% than other models. After the prediction, we used the predicted score and the oxygenated hemoglobin to observe cognitive effort by calculating relative neural efficiency and involvement in our test cases. Our result shows that even with moderate accuracy, the predicted cognitive effort closely follow the actual trends. This findings can be helpful in designing and improving learning environments and provide valuable insights into learning materials.


Efficient Pain Recognition via Respiration Signals: A Single Cross-Attention Transformer Multi-Window Fusion Pipeline

Gkikas, Stefanos, Kyprakis, Ioannis, Tsiknakis, Manolis

arXiv.org Artificial Intelligence

Pain is a complex condition that affects a large portion of the population. Accurate and consistent evaluation is essential for individuals experiencing pain and supports the development of effective and advanced management strategies. Automatic pain assessment systems provide continuous monitoring, aid clinical decision-making, and aim to reduce distress while preventing functional decline. This study has been submitted to the Second Multimodal Sensing Grand Challenge for Next-Gen Pain Assessment (AI4PAIN). The proposed method introduces a pipeline that employs respiration as the input signal and integrates a highly efficient cross-attention transformer with a multi-windowing strategy. Extensive experiments demonstrate that respiration serves as a valuable physiological modality for pain assessment. Furthermore, results show that compact and efficient models, when properly optimized, can deliver strong performance, often surpassing larger counterparts. The proposed multi-window strategy effectively captures short-term and long-term features, along with global characteristics, enhancing the model's representational capacity.


Multi-Representation Diagrams for Pain Recognition: Integrating Various Electrodermal Activity Signals into a Single Image

Gkikas, Stefanos, Kyprakis, Ioannis, Tsiknakis, Manolis

arXiv.org Artificial Intelligence

Pain is a multifaceted phenomenon that affects a substantial portion of the population. Reliable and consistent evaluation supports individuals experiencing pain and enables the development of effective and advanced management strategies. Automatic pain-assessment systems provide continuous monitoring, guide clinical decision-making, and aim to reduce distress while preventing functional decline. Incorporating physiological signals allows these systems to deliver objective, accurate insights into an individual's condition. This study has been submitted to the Second Multimodal Sensing Grand Challenge for Next-Gen Pain Assessment (AI4PAIN). The proposed method introduces a pipeline that employs electrodermal activity signals as the input modality. Multiple signal representations are generated and visualized as waveforms, which are then jointly presented within a unified multi-representation diagram. Extensive experiments using diverse processing and filtering techniques, along with various representation combinations, highlight the effectiveness of the approach. It consistently achieves comparable and, in several cases, superior results to traditional fusion methods, positioning it as a robust alternative for integrating different signal representations or modalities.


Multimodal Behavioral Patterns Analysis with Eye-Tracking and LLM-Based Reasoning

Guo, Dongyang, Abdrabou, Yasmeen, Thaqi, Enkeleda, Kasneci, Enkelejda

arXiv.org Artificial Intelligence

Eye-tracking data reveals valuable insights into users' cognitive states but is difficult to analyze due to its structured, non-linguistic nature. While large language models (LLMs) excel at reasoning over text, they struggle with temporal and numerical data. This paper presents a multimodal human-AI collaborative framework designed to enhance cognitive pattern extraction from eye-tracking signals. The framework includes: (1) a multi-stage pipeline using horizontal and vertical segmentation alongside LLM reasoning to uncover latent gaze patterns; (2) an Expert-Model Co-Scoring Module that integrates expert judgment with LLM output to generate trust scores for behavioral interpretations; and (3) a hybrid anomaly detection module combining LSTM-based temporal modeling with LLM-driven semantic analysis. Our results across several LLMs and prompt strategies show improvements in consistency, interpretability, and performance, with up to 50% accuracy in difficulty prediction tasks. This approach offers a scalable, interpretable solution for cognitive modeling and has broad potential in adaptive learning, human-computer interaction, and educational analytics.


Analyzing Transformers in Embedding Space

Dar, Guy, Geva, Mor, Gupta, Ankit, Berant, Jonathan

arXiv.org Artificial Intelligence

Understanding Transformer-based models has attracted significant attention, as they lie at the heart of recent technological advances across machine learning. While most interpretability methods rely on running models over inputs, recent work has shown that a zero-pass approach, where parameters are interpreted directly without a forward/backward pass is feasible for some Transformer parameters, and for two-layer attention networks. In this work, we present a theoretical analysis where all parameters of a trained Transformer are interpreted by projecting them into the embedding space, that is, the space of vocabulary items they operate on. We derive a simple theoretical framework to support our arguments and provide ample evidence for its validity. First, an empirical analysis showing that parameters of both pretrained and fine-tuned models can be interpreted in embedding space. Second, we present two applications of our framework: (a) aligning the parameters of different models that share a vocabulary, and (b) constructing a classifier without training by ``translating'' the parameters of a fine-tuned classifier to parameters of a different model that was only pretrained. Overall, our findings open the door to interpretation methods that, at least in part, abstract away from model specifics and operate in the embedding space only.


Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph

Sun, Jiashuo, Xu, Chengjin, Tang, Lumingyuan, Wang, Saizhuo, Lin, Chen, Gong, Yeyun, Ni, Lionel M., Shum, Heung-Yeung, Guo, Jian

arXiv.org Artificial Intelligence

Although large language models (LLMs) have achieved significant success in various tasks, they often struggle with hallucination problems, especially in scenarios requiring deep and responsible reasoning. These issues could be partially addressed by introducing external knowledge graphs (KG) in LLM reasoning. In this paper, we propose a new LLM-KG integrating paradigm ``$\hbox{LLM}\otimes\hbox{KG}$'' which treats the LLM as an agent to interactively explore related entities and relations on KGs and perform reasoning based on the retrieved knowledge. We further implement this paradigm by introducing a new approach called Think-on-Graph (ToG), in which the LLM agent iteratively executes beam search on KG, discovers the most promising reasoning paths, and returns the most likely reasoning results. We use a number of well-designed experiments to examine and illustrate the following advantages of ToG: 1) compared with LLMs, ToG has better deep reasoning power; 2) ToG has the ability of knowledge traceability and knowledge correctability by leveraging LLMs reasoning and expert feedback; 3) ToG provides a flexible plug-and-play framework for different LLMs, KGs and prompting strategies without any additional training cost; 4) the performance of ToG with small LLM models could exceed large LLM such as GPT-4 in certain scenarios and this reduces the cost of LLM deployment and application. As a training-free method with lower computational cost and better generality, ToG achieves overall SOTA in 6 out of 9 datasets where most previous SOTAs rely on additional training.


KEYword based Sampling (KEYS) for Large Language Models

S, Jyothir V, Akhtar, Zuhaib

arXiv.org Artificial Intelligence

Question answering (Q/A) can be formulated as a generative task (Mitra, 2017) where the task is to generate an answer given the question and the passage (knowledge, if available). Recent advances in QA task is focused a lot on language model advancements and less on other areas such as sampling(Krishna et al., 2021), (Nakano et al., 2021). Keywords play very important role for humans in language generation. (Humans formulate keywords and use grammar to connect those keywords and work). In the research community, very little focus is on how humans generate answers to a question and how this behavior can be incorporated in a language model. In this paper, we want to explore these two areas combined, i.e., how sampling can be to used generate answers which are close to human-like behavior and factually correct. Hence, the type of decoding algorithm we think should be used for Q/A tasks should also depend on the keywords. These keywords can be obtained from the question, passage or internet results. We use knowledge distillation techniques to extract keywords and sample using these extracted keywords on top of vanilla decoding algorithms when formulating the answer to generate a human-like answer. In this paper, we show that our decoding method outperforms most commonly used decoding methods for Q/A task


After years of fanfare the future of drone delivery in Australia remains up in the air

The Guardian

In 2013, Jeff Bezos announced Amazon was developing a drone delivery service. He estimated at the time that air-dropped packages were "four, five years" away. Nearly a decade later, the service is promised to begin by the end of this year – albeit in only two locations in the US. According to David Carbon, an Australian expat and vice-president of the firm's drone delivery division, Amazon wants to deliver 500m packages annually by drone from 2030. Carbon told AAP earlier this month that the firm was planning a wider rollout for air deliveries in the US and potentially Australia.


When Algorithms Rule, Values Can Wither

#artificialintelligence

Interest in the possibilities afforded by algorithms and big data continues to blossom as early adopters gain benefits from AI systems that automate decisions as varied as making customer recommendations, screening job applicants, detecting fraud, and optimizing logistical routes.1 But when AI applications fail, they can do so quite spectacularly.2 Consider the recent example of Australia's "robodebt" scandal.3 In 2015, the Australian government established its Income Compliance Program, with the goal of clawing back unemployment and disability benefits that had been made inappropriately to recipients. It set out to identify overpayments by analyzing discrepancies between the annual income that individuals reported and the income assessed by the Australian Tax Office.


Little Ripper deploys croc-spotting AI drones ZDNet

#artificialintelligence

The same artificial intelligence (AI) drone technology that the Little Ripper Group used for its shark detection drones is now being used to spot crocodiles in Queensland. Little Ripper Group co-founder Paul Scully-Power said the company was approached by the Queensland government to help keep beachgoers safe in the water and on land from crocodiles. "The Queensland government said, 'Hey do we have a challenge for you and asked can you spot crocodiles for us?' Crocodiles are slinky people that like dark, muddy water, so we took on that challenge," he said. The launch of the crocodile-spotting drones follows on from a trial that was carried out between Surf Life Saving Queensland and the Little Ripper Group to identify, monitor, and track the movement of crocodiles in November. The drone technology, dubbed the Little Ripper and designed together with the University of Technology Sydney, uses an AI system that was originally designed to detect sharks in real-time.