Instructional Material
The EAP-AIAS: Adapting the AI Assessment Scale for English for Academic Purposes
Roe, Jasper, Perkins, Mike, Tregubova, Yulia
The rapid advancement of Generative Artificial Intelligence (GenAI) presents both opportunities and challenges for English for Academic Purposes (EAP) instruction. This paper proposes an adaptation of the AI Assessment Scale (AIAS) specifically tailored for EAP contexts, termed the EAP-AIAS. This framework aims to provide a structured approach for integrating GenAI tools into EAP assessment practices while maintaining academic integrity and supporting language development. The EAP-AIAS consists of five levels, ranging from "No AI" to "Full AI", each delineating appropriate GenAI usage in EAP tasks. We discuss the rationale behind this adaptation, considering the unique needs of language learners and the dual focus of EAP on language proficiency and academic acculturation. This paper explores potential applications of the EAP-AIAS across various EAP assessment types, including writing tasks, presentations, and research projects. By offering a flexible framework, the EAP-AIAS seeks to empower EAP practitioners seeking to deal with the complexities of GenAI integration in education and prepare students for an AI-enhanced academic and professional future. This adaptation represents a step towards addressing the pressing need for ethical and pedagogically sound AI integration in language education.
Evaluating the Impact of Advanced LLM Techniques on AI-Lecture Tutors for a Robotics Course
Kahl, Sebastian, Löffler, Felix, Maciol, Martin, Ridder, Fabian, Schmitz, Marius, Spanagel, Jennifer, Wienkamp, Jens, Burgahn, Christopher, Schilling, Malte
This study evaluates the performance of Large Language Models (LLMs) as an Artificial Intelligence-based tutor for a university course. In particular, different advanced techniques are utilized, such as prompt engineering, Retrieval-Augmented-Generation (RAG), and fine-tuning. We assessed the different models and applied techniques using common similarity metrics like BLEU-4, ROUGE, and BERTScore, complemented by a small human evaluation of helpfulness and trustworthiness. Our findings indicate that RAG combined with prompt engineering significantly enhances model responses and produces better factual answers. In the context of education, RAG appears as an ideal technique as it is based on enriching the input of the model with additional information and material which usually is already present for a university course. Fine-tuning, on the other hand, can produce quite small, still strong expert models, but poses the danger of overfitting. Our study further asks how we measure performance of LLMs and how well current measurements represent correctness or relevance? We find high correlation on similarity metrics and a bias of most of these metrics towards shorter responses. Overall, our research points to both the potential and challenges of integrating LLMs in educational settings, suggesting a need for balanced training approaches and advanced evaluation frameworks.
Counterfactual Explanations for Medical Image Classification and Regression using Diffusion Autoencoder
Atad, Matan, Schinz, David, Moeller, Hendrik, Graf, Robert, Wiestler, Benedikt, Rueckert, Daniel, Navab, Nassir, Kirschke, Jan S., Keicher, Matthias
Counterfactual explanations (CEs) aim to enhance the interpretability of machine learning models by illustrating how alterations in input features would affect the resulting predictions. Common CE approaches require an additional model and are typically constrained to binary counterfactuals. In contrast, we propose a novel method that operates directly on the latent space of a generative model, specifically a Diffusion Autoencoder (DAE). This approach offers inherent interpretability by enabling the generation of CEs and the continuous visualization of the model's internal representation across decision boundaries. Our method leverages the DAE's ability to encode images into a semantically rich latent space in an unsupervised manner, eliminating the need for labeled data or separate feature extraction models. We show that these latent representations are helpful for medical condition classification and the ordinal regression of severity pathologies, such as vertebral compression fractures (VCF) and diabetic retinopathy (DR). Beyond binary CEs, our method supports the visualization of ordinal CEs using a linear model, providing deeper insights into the model's decision-making process and enhancing interpretability. Experiments across various medical imaging datasets demonstrate the method's advantages in interpretability and versatility. The linear manifold of the DAE's latent space allows for meaningful interpolation and manipulation, making it a powerful tool for exploring medical image properties. Our code is available at https://github.com/matanat/dae_counterfactual.
Integrating Cognitive AI with Generative Models for Enhanced Question Answering in Skill-based Learning
Madhusudhana, Rochan H., Dass, Rahul K., Luu, Jeanette, Goel, Ashok K.
In online learning, the ability to provide quick and accurate feedback to learners is crucial. In skill-based learning, learners need to understand the underlying concepts and mechanisms of a skill to be able to apply it effectively. While videos are a common tool in online learning, they cannot comprehend or assess the skills being taught. Additionally, while Generative AI methods are effective in searching and retrieving answers from a text corpus, it remains unclear whether these methods exhibit any true understanding. This limits their ability to provide explanations of skills or help with problem-solving. This paper proposes a novel approach that merges Cognitive AI and Generative AI to address these challenges. We employ a structured knowledge representation, the TMK (Task-Method-Knowledge) model, to encode skills taught in an online Knowledge-based AI course. Leveraging techniques such as Large Language Models, Chain-of-Thought, and Iterative Refinement, we outline a framework for generating reasoned explanations in response to learners' questions about skills.
Artifical intelligence and inherent mathematical difficulty
This paper explores the relationship of artificial intelligence to the task of resolving open questions in mathematics. We first present an updated version of a traditional argument that limitative results from computability and complexity theory show that proof discovery is an inherently difficult problem. We then illustrate how several recent applications of artificial intelligenceinspired methods - respectively involving automated theorem proving, Satsolvers, and large language models - do indeed raise novel questions about the nature of mathematical proof. We also argue that the results obtained by such techniques do not tell against our basic argument. This is so because they are embodiments of brute force search and are thus capable of deciding only statements of low logical complexity. Suppose... that we could find a finite system of rules which enabled us to say whether any given formula was demonstrable or not. This system would embody a theorem of metamathematics. There is of course no such theorem and this is very fortunate, since if there were we should have a mechanical set of rules for the solution of all mathematical problems, and our activities as mathematicians would come to an end.
CultureVo: The Serious Game of Utilizing Gen AI for Enhancing Cultural Intelligence
Agarwala, Ajita, Purwar, Anupam, Rao, Viswanadhasai
CultureVo, Inc. has developed the Integrated Culture Learning Suite (ICLS) to deliver foundational knowledge of world cultures through a combination of interactive lessons and gamified experiences. This paper explores how Generative AI powered by open source Large Langauge Models are utilized within the ICLS to enhance cultural intelligence. The suite employs Generative AI techniques to automate the assessment of learner knowledge, analyze behavioral patterns, and manage interactions with non-player characters using real time learner assessment. Additionally, ICLS provides contextual hint and recommend course content by assessing learner proficiency, while Generative AI facilitates the automated creation and validation of educational content.
Multimodal Fusion and Coherence Modeling for Video Topic Segmentation
Yu, Hai, Deng, Chong, Zhang, Qinglin, Liu, Jiaqing, Chen, Qian, Wang, Wen
Also, coherence is essential for data (Koshorek et al., 2018; Arnold et al., 2019), understanding logical structures and semantics. Enhancing contemporary supervised models (Lukasik et al., coherence modeling has achieved significant 2020; Somasundaran et al., 2020; Zhang et al., improvements in long document topic segmentation 2021; Yu et al., 2023) have demonstrated superior (Yu et al., 2023). Therefore, we improve performance compared to unsupervised approaches supervised VTS methods by thoroughly exploring (Riedl and Biemann, 2012; Solbiati et al., multimodal fusion and multimodal coherence 2021). Notably, supervised models that excel at modeling. We enhance multimodal fusion modeling long sequences (Zhang et al., 2021; Yu from the perspectives of model architecture and et al., 2023) are capable of capturing longer contextual pre-training and fine-tuning tasks. Specifically, we nuances and thereby achieve better topic segmentation compare various multimodal fusion architectures performance, compared to models that built upon Cross-Attention and Mixture-of-Experts model local sentence pairs or block pairs (Wang (MoE). We investigate the effect of multi-modal et al., 2017; Lukasik et al., 2020). In addition, contrastive learning for general pre-training and recent works (Somasundaran et al., 2020; Xing fine-tuning for strengthening cross-modal alignment.
Towards Scalable GPU-Accelerated SNN Training via Temporal Fusion
Li, Yanchen, Li, Jiachun, Sun, Kebin, Leng, Luziwei, Cheng, Ran
Drawing on the intricate structures of the brain, Spiking Neural Networks (SNNs) emerge as a transformative development in artificial intelligence, closely emulating the complex dynamics of biological neural networks. While SNNs show promising efficiency on specialized sparse-computational hardware, their practical training often relies on conventional GPUs. This reliance frequently leads to extended computation times when contrasted with traditional Artificial Neural Networks (ANNs), presenting significant hurdles for advancing SNN research. To navigate this challenge, we present a novel temporal fusion method, specifically designed to expedite the propagation dynamics of SNNs on GPU platforms, which serves as an enhancement to the current significant approaches for handling deep learning tasks with SNNs. This method underwent thorough validation through extensive experiments in both authentic training scenarios and idealized conditions, confirming its efficacy and adaptability for single and multi-GPU systems. Benchmarked against various existing SNN libraries/implementations, our method achieved accelerations ranging from 5 to 40 on NVIDIA A100 GPUs.
An effect analysis of the balancing techniques on the counterfactual explanations of student success prediction models
Cavus, Mustafa, Kuzilek, Jakub
In the past decade, we have experienced a massive boom in the usage of digital solutions in higher education. Due to this boom, large amounts of data have enabled advanced data analysis methods to support learners and examine learning processes. One of the dominant research directions in learning analytics is predictive modeling of learners' success using various machine learning methods. To build learners' and teachers' trust in such methods and systems, exploring the methods and methodologies that enable relevant stakeholders to deeply understand the underlying machine-learning models is necessary. In this context, counterfactual explanations from explainable machine learning tools are promising. Several counterfactual generation methods hold much promise, but the features must be actionable and causal to be effective. Thus, obtaining which counterfactual generation method suits the student success prediction models in terms of desiderata, stability, and robustness is essential. Although a few studies have been published in recent years on the use of counterfactual explanations in educational sciences, they have yet to discuss which counterfactual generation method is more suitable for this problem. This paper analyzed the effectiveness of commonly used counterfactual generation methods, such as WhatIf Counterfactual Explanations, Multi-Objective Counterfactual Explanations, and Nearest Instance Counterfactual Explanations after balancing. This contribution presents a case study using the Open University Learning Analytics dataset to demonstrate the practical usefulness of counterfactual explanations. The results illustrate the method's effectiveness and describe concrete steps that could be taken to alter the model's prediction.
What's coming up at #IJCAI2024?
The 33rd International Joint Conference on Artificial Intelligence (IJCAI-24) will be held in Jeju Island, South Korea from 3-9 August. The programme will feature keynote talks, panel discussions, tutorial, workshops, and oral and poster presentations. There will also be three special tracks, focussing on: AI, arts and creativity, AI for social good, and human-centred AI. There are six invited keynotes at this year's conference. These will be delivered from 6-9 August.