Education
SymbioSim: Human-in-the-loop Simulation Platform for Bidirectional Continuing Learning in Human-Robot Interaction
Chen, Haoran, Xu, Yiteng, Ren, Yiming, Ye, Yaoqin, Li, Xinran, Ding, Ning, Cong, Peishan, Wang, Ziyi, Liu, Bushi, Chen, Yuhan, Dou, Zhiyang, Leng, Xiaokun, Li, Manyi, Ma, Yuexin, Tu, Changhe
The development of intelligent robots seeks to seamlessly integrate them into the human world, providing assistance and companionship in daily life and work, with the ultimate goal of achieving human-robot symbiosis. To realize this vision, robots must continuously learn and evolve through consistent interaction and collaboration with humans, while humans need to gradually develop an understanding of and trust in robots through shared experiences. However, training and testing algorithms directly on physical robots involve substantial costs and safety risks. Moreover, current robotic simulators fail to support real human participation, limiting their ability to provide authentic interaction experiences and gather valuable human feedback. In this paper, we introduce SymbioSim, a novel human-in-the-loop robotic simulation platform designed to enable the safe and efficient development, evaluation, and optimization of human-robot interactions. By leveraging a carefully designed system architecture and modules, SymbioSim delivers a natural and realistic interaction experience, facilitating bidirectional continuous learning and adaptation for both humans and robots. Extensive experiments and user studies demonstrate the platform's promising performance and highlight its potential to significantly advance research on human-robot symbiosis.
Coupling Agent-Based Simulations and VR universes: the case of GAMA and Unity
Drogoul, Alexis, Taillandier, Patrick, Brugiรจre, Arthur, Martinez, Louis, Sillano, Lรฉon, Lesquoy, Baptiste, Nghi, Huynh Quang
Agent-based models (ABMs) and video games, including those taking advantage of virtual reality (VR), have undergone a remarkable parallel evolution, achieving impressive levels of complexity and sophistication. This paper argues that while ABMs prioritize scientific analysis and understanding and VR aims for immersive entertainment, they both simulate artificial worlds and can benefit from closer integration. Coupling both approaches indeed opens interesting possibilities for research and development in various fields, and in particular education, at the heart of the SIMPLE project, an EU-funded project on the development of digital tools for awareness raising on environmental issues. However, existing tools often present limitations, including technical complexity, limited functionalities, and lack of interoperability. To address these challenges, we introduce a novel framework for linking GAMA, a popular ABM platform, with Unity, a widely used game engine. This framework enables seamless data exchange, real-time visualization, and user interaction within VR environments, allowing researchers to leverage the strengths of both ABMs and VR for more impactful and engaging simulations. We demonstrate the capabilities of our framework through two prototypes built to highlight its potential in representing and interacting with complex socio-environmental system models. We conclude by emphasizing the importance of continued collaboration between the ABM and VR communities to develop robust, user-friendly tools, paving the way for a new era of collaborative research and immersive experiences in simulations.
UKTA: Unified Korean Text Analyzer
Ahn, Seokho, Park, Junhyung, Go, Ganghee, Kim, Chulhui, Jung, Jiho, Shin, Myung Sun, Kim, Do-Guk, Seo, Young-Duk
High-level, abstract evaluation results should be interpretable by humans, who need to understand Evaluating writing quality is complex and time-consuming often the reason behind the scores and the features that influenced the delaying feedback to learners. While automated writing evaluation results. Providing this explainability to users is crucial for ensuring tools are effective for English, Korean automated writing evaluation reliability, as these tools have the potential to make mistakes; tools face challenges due to their inability to address multi-view Unfortunately, existing Korean text analyzers [16, 18, 20] and automated analysis, error propagation, and evaluation explainability. To overcome writing evaluation tools [21, 37] do not fully meet all these these challenges, we introduce UKTA (Unified Korean Text requirements, limiting their practical use. Analyzer), a comprehensive Korea text analysis and writing evaluation To address the research gap, we introduce UKTA (Unified Korean system. UKTA provides accurate low-level morpheme analysis, Text Analyzer), a comprehensive Korean text analysis system for key lexical features for mid-level explainability, and transparent evaluating Korean writing. First, we provide accurate low-level analysis high-level rubric-based writing scores. Our approach enhances based on state-of-the-art Korean morpheme analyzer, which accuracy and quadratic weighted kappa over existing baseline, positioning minimizes error propagation. In addition to morpheme analysis, we UKTA as a leading multi-perspective tool for Korean text categorize and provide key features, such as lexical richness and analysis and writing evaluation.
KABB: Knowledge-Aware Bayesian Bandits for Dynamic Expert Coordination in Multi-Agent Systems
Zhang, Jusheng, Huang, Zimeng, Fan, Yijia, Liu, Ningyuan, Li, Mingyan, Yang, Zhuojie, Yao, Jiawei, Wang, Jian, Wang, Keze
As scaling large language models faces prohibitive costs, multi-agent systems emerge as Multi-Agent Systems (MAS) (Guo et al., 2024b) offer a a promising alternative, though challenged by promising alternative by coordinating multiple specialized static knowledge assumptions and coordination agents to achieve superior performance compared to individual inefficiencies. We introduce Knowledge-Aware systems while maintaining manageable computational Bayesian Bandits (KABB), a novel framework costs and budgets. Recent advances in MAS have led to that enhances multi-agent system coordination the development of several frameworks. For example, the through semantic understanding and dynamic Mixture of Agents (MoA) (Wang et al., 2024) employs multiple adaptation. The framework features three key LLMs as proposers to iteratively refine responses, with innovations: a three-dimensional knowledge distance a central aggregator delivering the final output. Although model for deep semantic understanding, a MoA has demonstrated robustness and scalability in deployment, dual-adaptation mechanism for continuous expert its computational cost scales linearly with the number optimization, and a knowledge-aware Thompson of agents, and significant redundancy and noise become a Sampling strategy for efficient expert selection.
Cascading Bandits Robust to Adversarial Corruptions
Xie, Jize, Chen, Cheng, Wang, Zhiyong, Li, Shuai
Online learning to rank sequentially recommends a small list of items to users from a large candidate set and receives the users' click feedback. In many real-world scenarios, users browse the recommended list in order and click the first attractive item without checking the rest. Such behaviors are usually formulated as the cascade model. Many recent works study algorithms for cascading bandits, an online learning to rank framework in the cascade model. However, the performance of existing methods may drop significantly if part of the user feedback is adversarially corrupted (e.g., click fraud). In this work, we study how to resist adversarial corruptions in cascading bandits. We first formulate the ``\textit{Cascading Bandits with Adversarial Corruptions}" (CBAC) problem, which assumes that there is an adaptive adversary that may manipulate the user feedback. Then we propose two robust algorithms for this problem, which assume the corruption level is known and agnostic, respectively. We show that both algorithms can achieve logarithmic regret when the algorithm is not under attack, and the regret increases linearly with the corruption level. The experimental results also verify the robustness of our methods.
WorldGUI: Dynamic Testing for Comprehensive Desktop GUI Automation
Zhao, Henry Hengyuan, Gao, Difei, Shou, Mike Zheng
Current GUI agents have achieved outstanding performance in GUI element grounding. However, planning remains highly challenging, especially due to sensitivity to the initial state of the environment. Specifically, slight differences in the initial state-such as the target software not being open or the interface not being in its default state-often lead to planning errors. This issue is widespread in real user scenarios, but existing benchmarks fail to evaluate it. In this paper, we present WorldGUI, a novel GUI benchmark that designs GUI tasks with various initial states to simulate real computer-user interactions. The benchmark spans a wide range of tasks across 10 popular software applications, including PowerPoint, VSCode, and Adobe Acrobat. In addition, to address the challenges of dynamic GUI automation tasks, we propose GUI-Thinker, a holistic framework, leveraging a critique mechanism, that effectively manages the unpredictability and complexity of GUI interactions. Experimental results demonstrate that GUI-Thinker significantly outperforms Claude-3.5 (Computer Use) by 14.9% in success rate on WorldGUI tasks. This improvement underscores the effectiveness of our critical-thinking-based framework in enhancing GUI automation.
CREDAL: Close Reading of Data Models
Fletcher, George, Nahurna, Olha, Prytula, Matvii, Stoyanovich, Julia
Data models are necessary for the birth of data and of any data-driven system. Indeed, every algorithm, every machine learning model, every statistical model, and every database has an underlying data model without which the system would not be usable. Hence, data models are excellent sites for interrogating the (material, social, political, ...) conditions giving rise to a data system. Towards this, drawing inspiration from literary criticism, we propose to closely read data models in the same spirit as we closely read literary artifacts. Close readings of data models reconnect us with, among other things, the materiality, the genealogies, the techne, the closed nature, and the design of technical systems. While recognizing from literary theory that there is no one correct way to read, it is nonetheless critical to have systematic guidance for those unfamiliar with close readings. This is especially true for those trained in the computing and data sciences, who too often are enculturated to set aside the socio-political aspects of data work. A systematic methodology for reading data models currently does not exist. To fill this gap, we present the CREDAL methodology for close readings of data models. We detail our iterative development process and present results of a qualitative evaluation of CREDAL demonstrating its usability, usefulness, and effectiveness in the critical study of data.
Enhancing Higher Education with Generative AI: A Multimodal Approach for Personalised Learning
This research explores the opportunities of Generative AI (GenAI) in the realm of higher education through the design and development of a multimodal chatbot for an undergraduate course. Leveraging the ChatGPT API for nuanced text-based interactions and Google Bard for advanced image analysis and diagram-to-code conversions, we showcase the potential of GenAI in addressing a broad spectrum of educational queries. Additionally, the chatbot presents a file-based analyser designed for educators, offering deep insights into student feedback via sentiment and emotion analysis, and summarising course evaluations with key metrics. These combinations highlight the crucial role of multimodal conversational AI in enhancing teaching and learning processes, promising significant advancements in educational adaptability, engagement, and feedback analysis. By demonstrating a practical web application, this research underlines the imperative for integrating GenAI technologies to foster more dynamic and responsive educational environments, ultimately contributing to improved educational outcomes and pedagogical strategies.
TMLC-Net: Transferable Meta Label Correction for Noisy Label Learning
The prevalence of noisy labels in real-world datasets poses a significant impediment to the effective deployment of deep learning models. While meta-learning strategies have emerged as a promising approach for addressing this challenge, existing methods often suffer from limited transferability and task-specific designs. This paper introduces TMLC-Net, a novel Transferable Meta-Learner for Correcting Noisy Labels, designed to overcome these limitations. TMLC-Net learns a general-purpose label correction strategy that can be readily applied across diverse datasets and model architectures without requiring extensive retraining or fine-tuning. Our approach integrates three core components: (1) Normalized Noise Perception, which captures and normalizes training dynamics to handle distribution shifts; (2) Time-Series Encoding, which models the temporal evolution of sample statistics using a recurrent neural network; and (3) Subclass Decoding, which predicts a corrected label distribution based on the learned representations. We conduct extensive experiments on benchmark datasets with various noise types and levels, demonstrating that TMLC-Net consistently outperforms state-of-the-art methods in terms of both accuracy and robustness to label noise. Furthermore, we analyze the transferability of TMLC-Net, showcasing its adaptability to new datasets and noise conditions, and establishing its potential as a broadly applicable solution for robust deep learning in noisy environments.