Instructional Material
Fairness in Large Language Models in Three Hours
Viet, Thang Doan, Wang, Zichong, Nguyen, Minh Nhat, Zhang, Wenbin
For example, one line of work extends traditional fairness in LLMs involves unique backgrounds, taxonomies, and fairness notions--individual fairness and group fairness--to these fulfillment techniques. This tutorial provides a systematic overview models[6]. Specifically, individual fairness seeks to ensure similar of recent advances in the literature concerning fair LLMs, beginning outcomes for similar individuals [13, 49], while group fairness focuses with real-world case studies to introduce LLMs, followed by on equalizing outcome statistics across subgroups defined by an analysis of bias causes therein. The concept of fairness in LLMs sensitive attributes [18, 44-46] (e.g., gender or race). While these is then explored, summarizing the strategies for evaluating bias classification-based fairness notions are adept at evaluating bias in and the algorithms designed to promote fairness. Additionally, resources LLM's classification results[6], they fall short in addressing biases for assessing bias in LLMs, including toolkits and datasets, that arise during the LLM generation process[20].
Integrating HCI Datasets in Project-Based Machine Learning Courses: A College-Level Review and Case Study
Qu, Xiaodong, Key, Matthew, Luo, Eric, Qiu, Chuhui
This study explores the integration of real-world machine learning (ML) projects using human-computer interfaces (HCI) datasets in college-level courses to enhance both teaching and learning experiences. Employing a comprehensive literature review, course websites analysis, and a detailed case study, the research identifies best practices for incorporating HCI datasets into project-based ML education. Key findings demonstrate increased student engagement, motivation, and skill development through hands-on projects, while instructors benefit from effective tools for teaching complex concepts. The study also addresses challenges such as data complexity and resource allocation, offering recommendations for future improvements. These insights provide a valuable framework for educators aiming to bridge the gap between theoretical knowledge and practical application in ML education.
#IJCAI2024 โ tweet round-up of the tutorials and workshops
The 33rd International Joint Conference on Artificial Intelligence (IJCAI-24) is currently taking place in Jeju Island, South Korea. The first three days of the event saw the running of tutorials, workshops, and the doctorial consortium track. Find out what the participants have been getting up during these first few days. Welcome to our @IJCAIconf AIGC tutorial "Beyond Human Creativity: A Tutorial on Advancements in AI Generated Content" with @BangL93 @chenyu_hugo @hengjinlp @Teddy_LFWU Join us this afternoon! It was an honor to be invited to speak at #IJCAI2024 and introduce the concepts behind #timeseries databases to an amazing group of researchers and academics.
From Stem to Stern: Contestability Along AI Value Chains
Balayn, Agathe, Pi, Yulu, Widder, David Gray, Alfrink, Kars, Yurrita, Mireia, Upadhyay, Sohini, Karusala, Naveena, Lyons, Henrietta, Turkay, Cagatay, Tessono, Christelle, Attard-Frost, Blair, Gadiraju, Ujwal
This workshop will grow and consolidate a community of interdisciplinary CSCW researchers focusing on the topic of contestable AI. As an outcome of the workshop, we will synthesize the most pressing opportunities and challenges for contestability along AI value chains in the form of a research roadmap. This roadmap will help shape and inspire imminent work in this field. Considering the length and depth of AI value chains, it will especially spur discussions around the contestability of AI systems along various sites of such chains. The workshop will serve as a platform for dialogue and demonstrations of concrete, successful, and unsuccessful examples of AI systems that (could or should) have been contested, to identify requirements, obstacles, and opportunities for designing and deploying contestable AI in various contexts. This will be held primarily as an in-person workshop, with some hybrid accommodation. The day will consist of individual presentations and group activities to stimulate ideation and inspire broad reflections on the field of contestable AI. Our aim is to facilitate interdisciplinary dialogue by bringing together researchers, practitioners, and stakeholders to foster the design and deployment of contestable AI.
The EAP-AIAS: Adapting the AI Assessment Scale for English for Academic Purposes
Roe, Jasper, Perkins, Mike, Tregubova, Yulia
The rapid advancement of Generative Artificial Intelligence (GenAI) presents both opportunities and challenges for English for Academic Purposes (EAP) instruction. This paper proposes an adaptation of the AI Assessment Scale (AIAS) specifically tailored for EAP contexts, termed the EAP-AIAS. This framework aims to provide a structured approach for integrating GenAI tools into EAP assessment practices while maintaining academic integrity and supporting language development. The EAP-AIAS consists of five levels, ranging from "No AI" to "Full AI", each delineating appropriate GenAI usage in EAP tasks. We discuss the rationale behind this adaptation, considering the unique needs of language learners and the dual focus of EAP on language proficiency and academic acculturation. This paper explores potential applications of the EAP-AIAS across various EAP assessment types, including writing tasks, presentations, and research projects. By offering a flexible framework, the EAP-AIAS seeks to empower EAP practitioners seeking to deal with the complexities of GenAI integration in education and prepare students for an AI-enhanced academic and professional future. This adaptation represents a step towards addressing the pressing need for ethical and pedagogically sound AI integration in language education.
Evaluating the Impact of Advanced LLM Techniques on AI-Lecture Tutors for a Robotics Course
Kahl, Sebastian, Lรถffler, Felix, Maciol, Martin, Ridder, Fabian, Schmitz, Marius, Spanagel, Jennifer, Wienkamp, Jens, Burgahn, Christopher, Schilling, Malte
This study evaluates the performance of Large Language Models (LLMs) as an Artificial Intelligence-based tutor for a university course. In particular, different advanced techniques are utilized, such as prompt engineering, Retrieval-Augmented-Generation (RAG), and fine-tuning. We assessed the different models and applied techniques using common similarity metrics like BLEU-4, ROUGE, and BERTScore, complemented by a small human evaluation of helpfulness and trustworthiness. Our findings indicate that RAG combined with prompt engineering significantly enhances model responses and produces better factual answers. In the context of education, RAG appears as an ideal technique as it is based on enriching the input of the model with additional information and material which usually is already present for a university course. Fine-tuning, on the other hand, can produce quite small, still strong expert models, but poses the danger of overfitting. Our study further asks how we measure performance of LLMs and how well current measurements represent correctness or relevance? We find high correlation on similarity metrics and a bias of most of these metrics towards shorter responses. Overall, our research points to both the potential and challenges of integrating LLMs in educational settings, suggesting a need for balanced training approaches and advanced evaluation frameworks.
Counterfactual Explanations for Medical Image Classification and Regression using Diffusion Autoencoder
Atad, Matan, Schinz, David, Moeller, Hendrik, Graf, Robert, Wiestler, Benedikt, Rueckert, Daniel, Navab, Nassir, Kirschke, Jan S., Keicher, Matthias
Counterfactual explanations (CEs) aim to enhance the interpretability of machine learning models by illustrating how alterations in input features would affect the resulting predictions. Common CE approaches require an additional model and are typically constrained to binary counterfactuals. In contrast, we propose a novel method that operates directly on the latent space of a generative model, specifically a Diffusion Autoencoder (DAE). This approach offers inherent interpretability by enabling the generation of CEs and the continuous visualization of the model's internal representation across decision boundaries. Our method leverages the DAE's ability to encode images into a semantically rich latent space in an unsupervised manner, eliminating the need for labeled data or separate feature extraction models. We show that these latent representations are helpful for medical condition classification and the ordinal regression of severity pathologies, such as vertebral compression fractures (VCF) and diabetic retinopathy (DR). Beyond binary CEs, our method supports the visualization of ordinal CEs using a linear model, providing deeper insights into the model's decision-making process and enhancing interpretability. Experiments across various medical imaging datasets demonstrate the method's advantages in interpretability and versatility. The linear manifold of the DAE's latent space allows for meaningful interpolation and manipulation, making it a powerful tool for exploring medical image properties. Our code is available at https://github.com/matanat/dae_counterfactual.
Integrating Cognitive AI with Generative Models for Enhanced Question Answering in Skill-based Learning
Madhusudhana, Rochan H., Dass, Rahul K., Luu, Jeanette, Goel, Ashok K.
In online learning, the ability to provide quick and accurate feedback to learners is crucial. In skill-based learning, learners need to understand the underlying concepts and mechanisms of a skill to be able to apply it effectively. While videos are a common tool in online learning, they cannot comprehend or assess the skills being taught. Additionally, while Generative AI methods are effective in searching and retrieving answers from a text corpus, it remains unclear whether these methods exhibit any true understanding. This limits their ability to provide explanations of skills or help with problem-solving. This paper proposes a novel approach that merges Cognitive AI and Generative AI to address these challenges. We employ a structured knowledge representation, the TMK (Task-Method-Knowledge) model, to encode skills taught in an online Knowledge-based AI course. Leveraging techniques such as Large Language Models, Chain-of-Thought, and Iterative Refinement, we outline a framework for generating reasoned explanations in response to learners' questions about skills.
Artifical intelligence and inherent mathematical difficulty
This paper explores the relationship of artificial intelligence to the task of resolving open questions in mathematics. We first present an updated version of a traditional argument that limitative results from computability and complexity theory show that proof discovery is an inherently difficult problem. We then illustrate how several recent applications of artificial intelligenceinspired methods - respectively involving automated theorem proving, Satsolvers, and large language models - do indeed raise novel questions about the nature of mathematical proof. We also argue that the results obtained by such techniques do not tell against our basic argument. This is so because they are embodiments of brute force search and are thus capable of deciding only statements of low logical complexity. Suppose... that we could find a finite system of rules which enabled us to say whether any given formula was demonstrable or not. This system would embody a theorem of metamathematics. There is of course no such theorem and this is very fortunate, since if there were we should have a mechanical set of rules for the solution of all mathematical problems, and our activities as mathematicians would come to an end.
CultureVo: The Serious Game of Utilizing Gen AI for Enhancing Cultural Intelligence
Agarwala, Ajita, Purwar, Anupam, Rao, Viswanadhasai
CultureVo, Inc. has developed the Integrated Culture Learning Suite (ICLS) to deliver foundational knowledge of world cultures through a combination of interactive lessons and gamified experiences. This paper explores how Generative AI powered by open source Large Langauge Models are utilized within the ICLS to enhance cultural intelligence. The suite employs Generative AI techniques to automate the assessment of learner knowledge, analyze behavioral patterns, and manage interactions with non-player characters using real time learner assessment. Additionally, ICLS provides contextual hint and recommend course content by assessing learner proficiency, while Generative AI facilitates the automated creation and validation of educational content.