Education
Scalable Learning of Item Response Theory Models
Frick, Susanne, Krivošija, Amer, Munteanu, Alexander
Item Response Theory (IRT) models aim to assess latent abilities of $n$ examinees along with latent difficulty characteristics of $m$ test items from categorical data that indicates the quality of their corresponding answers. Classical psychometric assessments are based on a relatively small number of examinees and items, say a class of $200$ students solving an exam comprising $10$ problems. More recent global large scale assessments such as PISA, or internet studies, may lead to significantly increased numbers of participants. Additionally, in the context of Machine Learning where algorithms take the role of examinees and data analysis problems take the role of items, both $n$ and $m$ may become very large, challenging the efficiency and scalability of computations. To learn the latent variables in IRT models from large data, we leverage the similarity of these models to logistic regression, which can be approximated accurately using small weighted subsets called coresets. We develop coresets for their use in alternating IRT training algorithms, facilitating scalable learning from large data.
Certain and Approximately Certain Models for Statistical Learning
Zhen, Cheng, Aryal, Nischal, Termehchy, Arash, Aghasi, Alireza, Chabada, Amandeep Singh
Real-world data is often incomplete and contains missing values. To train accurate models over real-world datasets, users need to spend a substantial amount of time and resources imputing and finding proper values for missing data items. In this paper, we demonstrate that it is possible to learn accurate models directly from data with missing values for certain training data and target models. We propose a unified approach for checking the necessity of data imputation to learn accurate models across various widely-used machine learning paradigms. We build efficient algorithms with theoretical guarantees to check this necessity and return accurate models in cases where imputation is unnecessary. Our extensive experiments indicate that our proposed algorithms significantly reduce the amount of time and effort needed for data imputation without imposing considerable computational overhead.
Case Studies of AI Policy Development in Africa
Diallo, Kadijatou, Smith, Jonathan, Okolo, Chinasa T., Nyamwaya, Dorcas, Kgomo, Jonas, Ngamita, Richard
Artificial Intelligence (AI) requires new ways of evaluating national technology use and strategy for African nations. We conduct a survey of existing 'readiness' assessments both for general digital adoption and for AI policy in particular. We conclude that existing global readiness assessments do not fully capture African states' progress in AI readiness and lay the groundwork for how assessments can be better used for the African context. We consider the extent to which these indicators map to the African context and what these indicators miss in capturing African states' on-the-ground work in meeting AI capability. Through case studies of four African nations of diverse geographic and economic dimensions, we identify nuances missed by global assessments and offer high-level policy considerations for how states can best improve their AI readiness standards and prepare their societies to capture the benefits of AI.
Towards Modeling Learner Performance with Large Language Models
Neshaei, Seyed Parsa, Davis, Richard Lee, Hazimeh, Adam, Lazarevski, Bojan, Dillenbourg, Pierre, Käser, Tanja
Recent work exploring the capabilities of pre-trained large language models (LLMs) has demonstrated their ability to act as general pattern machines by completing complex token sequences representing a wide array of tasks, including time-series prediction and robot control. This paper investigates whether the pattern recognition and sequence modeling capabilities of LLMs can be extended to the domain of knowledge tracing, a critical component in the development of intelligent tutoring systems (ITSs) that tailor educational experiences by predicting learner performance over time. In an empirical evaluation across multiple real-world datasets, we compare two approaches to using LLMs for this task, zero-shot prompting and model fine-tuning, with existing, non-LLM approaches to knowledge tracing. While LLM-based approaches do not achieve state-of-the-art performance, fine-tuned LLMs surpass the performance of naive baseline models and perform on par with standard Bayesian Knowledge Tracing approaches across multiple metrics. These findings suggest that the pattern recognition capabilities of LLMs can be used to model complex learning trajectories, opening a novel avenue for applying LLMs to educational contexts. The paper concludes with a discussion of the implications of these findings for future research, suggesting that further refinements and a deeper understanding of LLMs' predictive mechanisms could lead to enhanced performance in knowledge tracing tasks.
Learning to walk in confined spaces using 3D representation
Miki, Takahiro, Lee, Joonho, Wellhausen, Lorenz, Hutter, Marco
Legged robots have the potential to traverse complex terrain and access confined spaces beyond the reach of traditional platforms thanks to their ability to carefully select footholds and flexibly adapt their body posture while walking. However, robust deployment in real-world applications is still an open challenge. In this paper, we present a method for legged locomotion control using reinforcement learning and 3D volumetric representations to enable robust and versatile locomotion in confined and unstructured environments. By employing a two-layer hierarchical policy structure, we exploit the capabilities of a highly robust low-level policy to follow 6D commands and a high-level policy to enable three-dimensional spatial awareness for navigating under overhanging obstacles. Our study includes the development of a procedural terrain generator to create diverse training environments. We present a series of experimental evaluations in both simulation and real-world settings, demonstrating the effectiveness of our approach in controlling a quadruped robot in confined, rough terrain. By achieving this, our work extends the applicability of legged robots to a broader range of scenarios.
Federated Linear Contextual Bandits with Heterogeneous Clients
Blaser, Ethan, Li, Chuanhao, Wang, Hongning
The demand for collaborative and private bandit learning across multiple agents is surging due to the growing quantity of data generated from distributed systems. Federated bandit learning has emerged as a promising framework for private, efficient, and decentralized online learning. However, almost all previous works rely on strong assumptions of client homogeneity, i.e., all participating clients shall share the same bandit model; otherwise, they all would suffer linear regret. This greatly restricts the application of federated bandit learning in practice. In this work, we introduce a new approach for federated bandits for heterogeneous clients, which clusters clients for collaborative bandit learning under the federated learning setting. Our proposed algorithm achieves non-trivial sub-linear regret and communication cost for all clients, subject to the communication protocol under federated learning that at anytime only one model can be shared by the server.
SAIE Framework: Support Alone Isn't Enough -- Advancing LLM Training with Adversarial Remarks
Loem, Mengsay, Kaneko, Masahiro, Okazaki, Naoaki
Large Language Models (LLMs) can justify or critique their predictions through discussions with other models or humans, thereby enriching their intrinsic understanding of instances. While proactive discussions in the inference phase have been shown to boost performance, such interactions have not been extensively explored during the training phase. We hypothesize that incorporating interactive discussions into the training process can enhance the models' understanding and improve their reasoning and verbal expression abilities during inference. This work introduces the SAIE framework, which facilitates supportive and adversarial discussions between learner and partner models. The learner model receives responses from the partner, and its parameters are then updated based on this discussion. This dynamic adjustment process continues throughout the training phase, responding to the evolving outputs of the learner model. Our empirical evaluation across various tasks, including math problems, commonsense reasoning, and multi-domain knowledge, demonstrates that models fine-tuned with the SAIE framework outperform those trained with conventional fine-tuning approaches. Furthermore, our method enhances the models' reasoning capabilities, improving both individual and multi-agent inference performance.
What's in a Name? Auditing Large Language Models for Race and Gender Bias
Haim, Amit, Salinas, Alejandro, Nyarko, Julian
Large Language Models (LLM) have dramatically surged in popularity over the recent years. Since the release of ChatGPT, LLMs - especially those with an accessible chat interface - have not only been used by experts, but are also becoming an increasingly common tool with significant benefits for laypeople. To that end, many commercial actors have already begun implementing LLMs in their operations, ranging from customer-facing chatbots to internal decision support systems [14, 6]. The fairness of AI algorithms, including LLMs, has been a pernicious issue, motivating a growing literature and community of AI ethics research [8]. Disparities across gender and race, among other attributes, have especially preoccupied this field [4], leading to efforts to include bias auditing as an important component of AI harm mitigation in policy discussions and regulatory frameworks [28]. Mitigating biases arising from the explicit use of race or gender in the prompt is comparatively straightforward.
HearHere: Mitigating Echo Chambers in News Consumption through an AI-based Web System
Jeon, Youngseung, Kim, Jaehoon, Park, Sohyun, Ko, Yunyong, Ryu, Seongeun, Kim, Sang-Wook, Han, Kyungsik
This practice can lead to more rational decision-making that is not heavily influenced by specific opinions or positions [12, 22, 23]. As the Internet is a primary source of information for many people and the volume of online information is immense, effectively helping people consume and share information from diverse perspectives is necessary but challenging [57, 93]. Researchers have proposed various support methods for this, including the development and use of computer technology. In particular, artificial intelligence (AI)-based recommendation systems have been designed to support efficient information consumption by learning users' demographic characteristics or online activity patterns and providing tailored information based on their preferences [77]. Although computer technology plays an important role in enabling people to access and share online information, it should be noted that providing information solely based on individuals' preferences and tendencies can inadvertently contribute to the formation of echo chambers [77], a phenomenon where individuals are exposed primarily to the like-minded groups or information, leading to a reinforcement of shared narratives [28]. Research has shown that echo chambers can have many negative outcomes, including the creation and dissemination of biased information [77], increased susceptibility to fake news [8, 27], resistance towards accepting scientific evidence [63], and the adoption of unbalanced perspectives [36]. To prevent users from becoming polarized towards a specific political stance, many studies have proposed the use of computer-based tools designed to present information from diverse perspectives [31, 48, 53, 62].
A machine learning approach to predict university enrolment choices through students' high school background in Italy
Priulla, Andrea, Albano, Alessandro, D'Angelo, Nicoletta, Attanasio, Massimo
This paper explores the influence of Italian high school students' proficiency in mathematics and the Italian language on their university enrolment choices, specifically focusing on STEM (Science, Technology, Engineering, and Mathematics) courses. We distinguish between students from scientific and humanistic backgrounds in high school, providing valuable insights into their enrolment preferences. Furthermore, we investigate potential gender differences in response to similar previous educational choices and achievements. The study employs gradient boosting methodology, known for its high predicting performance and ability to capture non-linear relationships within data, and adjusts for variables related to the socio-demographic characteristics of the students and their previous educational achievements. Our analysis reveals significant differences in the enrolment choices based on previous high school achievements. The findings shed light on the complex interplay of academic proficiency, gender, and high school background in shaping students' choices regarding university education, with implications for educational policy and future research endeavours.