Generative AI
On the logical skills of large language models: evaluations using arbitrarily complex first-order logic problems
Ibragimov, Shokhrukh, Jentzen, Arnulf, Kuckuck, Benno
We present a method of generating first-order logic statements whose complexity can be controlled along multiple dimensions. We use this method to automatically create several datasets consisting of questions asking for the truth or falsity of first-order logic statements in Zermelo-Fraenkel set theory. While the resolution of these questions does not require any knowledge beyond basic notation of first-order logic and set theory, it does require a degree of planning and logical reasoning, which can be controlled up to arbitrarily high difficulty by the complexity of the generated statements. Furthermore, we do extensive evaluations of the performance of various large language models, including recent models such as DeepSeek-R1 and OpenAI's o3-mini, on these datasets. All of the datasets along with the code used for generating them, as well as all data from the evaluations is publicly available at https://github.com/bkuckuck/logical-skills-of-llms.
UM_FHS at TREC 2024 PLABA: Exploration of Fine-tuning and AI agent approach for plain language adaptations of biomedical text
Kocbek, Primoz, Kopitar, Leon, Zhang, Zhihong, Aydin, Emirhan, Topaz, Maxim, Stiglic, Gregor
This paper describes our submissions to the TREC 2024 PLABA track with the aim to simplify biomedical abstracts for a K8 - level audience (13 - 14 years old students). We tested three approaches using OpenAI's gpt - 4o and gpt - 4o - mini models: baseline prompt engineering, a two - AI agent approach, and fine - tuning. Adaptations were evaluated using qualitative metrics ( 5 - point Likert scales for simplicity, accuracy, completeness, and brevity) and quantitative readability scores (Flesch - Kincaid grade level, SMOG Index). Results indicate d that the two - agent approach and baseline prompt engineering with gpt - 4o - mini models show superior qualitative performance, while fine - tuned models excelled in accuracy and completeness but were less simple. The evaluation results demonstrated that prompt engineering with gpt - 4o - mini outperforms iterative improvement strategies via two - agent approach as well as fine - tuning with gpt - 4o. We intend to expand our investigation of the results and explore advanced evaluations.
Multi-Agent Risks from Advanced AI
Hammond, Lewis, Chan, Alan, Clifton, Jesse, Hoelscher-Obermaier, Jason, Khan, Akbir, McLean, Euan, Smith, Chandler, Barfuss, Wolfram, Foerster, Jakob, Gavenčiak, Tomáš, Han, The Anh, Hughes, Edward, Kovařík, Vojtěch, Kulveit, Jan, Leibo, Joel Z., Oesterheld, Caspar, de Witt, Christian Schroeder, Shah, Nisarg, Wellman, Michael, Bova, Paolo, Cimpeanu, Theodor, Ezell, Carson, Feuillade-Montixi, Quentin, Franklin, Matija, Kran, Esben, Krawczuk, Igor, Lamparth, Max, Lauffer, Niklas, Meinke, Alexander, Motwani, Sumeet, Reuel, Anka, Conitzer, Vincent, Dennis, Michael, Gabriel, Iason, Gleave, Adam, Hadfield, Gillian, Haghtalab, Nika, Kasirzadeh, Atoosa, Krier, Sébastien, Larson, Kate, Lehman, Joel, Parkes, David C., Piliouras, Georgios, Rahwan, Iyad
The rapid development of advanced AI agents and the imminent deployment of many instances of these agents will give rise to multi-agent systems of unprecedented complexity. These systems pose novel and under-explored risks. In this report, we provide a structured taxonomy of these risks by identifying three key failure modes (miscoordination, conflict, and collusion) based on agents' incentives, as well as seven key risk factors (information asymmetries, network effects, selection pressures, destabilising dynamics, commitment problems, emergent agency, and multi-agent security) that can underpin them. We highlight several important instances of each risk, as well as promising directions to help mitigate them. By anchoring our analysis in a range of real-world examples and experimental evidence, we illustrate the distinct challenges posed by multi-agent systems and their implications for the safety, governance, and ethics of advanced AI.
Personalized Education with Generative AI and Digital Twins: VR, RAG, and Zero-Shot Sentiment Analysis for Industry 4.0 Workforce Development
Lin, Yu-Zheng, Petal, Karan, Alhamadah, Ahmed H, Ghimire, Sujan, Redondo, Matthew William, Corona, David Rafael Vidal, Pacheco, Jesus, Salehi, Soheil, Satam, Pratik
While the advent of the Fourth Industrial Revolution (4IR) technologies, like cloud computing, machine learning, and artificial intelligence have brought convenience and productivity improvements, they have also introduced new challenges in training and education that require the reskilling of existing employees and the building of a new workforce. Exacerbated by the already existing workforce shortages, this mammoth workforce reskilling and building effort aims to build a high-tech workforce capable of operating and maintaining these 4IR systems; requiring a higher student retention and persistence. This increase in student retention and persistence will be especially critical when training the workforce originating from marginalized communities like Underrepresented Minorities (URM), where challenges arise due to lack of access to high-quality education throughout the trainee's formative years (pre/middle/high schools), creating a cyclic set of knowledge dependencies that are difficult to meet. To address these challenges, this research presents Generative AI-based Personalized Tutor for Industrial 4.0 (gAI-PT4I4), a framework that focuses on personalization of 4IR experiential learning, using sentiment analysis to gauge student's knowledge comprehension, while using a combination of generative AI and finite automaton to personalize the content to the students' learning needs. The framework administers experiential learning, using low-fidelity Digital Twins that enable virtual reality-based (VR) training exercises focusing on 4IR training. The VR environment, integrates a generative AI teaching assistant called the Interactive Tutor, that guides the student through the training exercises, with audio and text communications.
Human-Artificial Interaction in the Age of Agentic AI: A System-Theoretical Approach
Borghoff, Uwe M., Bottoni, Paolo, Pareschi, Remo
This paper presents a novel perspective on human-computer interaction (HCI), framing it as a dynamic interplay between human and computational agents within a networked system. Going beyond traditional interface-based approaches, we emphasize the importance of coordination and communication among heterogeneous agents with different capabilities, roles, and goals. A key distinction is made between multi-agent systems (MAS) and Centaurian systems, which represent two different paradigms of human-AI collaboration. MAS maintain agent autonomy, with structured protocols enabling cooperation, while Centau-rian systems deeply integrate human and AI capabilities, creating unified decision-making entities. To formalize these interactions, we introduce a framework for communication spaces, structured into surface, observation, and computation layers, ensuring seamless integration between MAS and Centaurian architectures, where colored Petri nets effectively represent structured Cen-taurian systems and high-level reconfigurable networks address the dynamic nature of MAS. Our research has practical applications in autonomous robotics, human-in-the-loop decision making, and AI-driven cognitive architectures, and provides a foundation for next-generation hybrid intelligence systems that balance structured coordination with emergent behavior. Keywords: multi-agent systems centaurian systems communication spaces satellite and swarm robots large action models (LAMs). 1 Introduction Agentic AI systems--capable of iterative planning, autonomous task decomposition, and continuous learning--are rapidly reshaping the landscape of human-computer interaction (HCI). Recent advances in Large Language Models (LLMs) and advanced conversational agents have revitalized the field of multi-agent systems, whose roots in Artificial Intelligence predate the current rise of generative AI. Historically, multi-agent systems relied on agents with relatively constrained capabilities; however, the emergence of powerful, conversationally Corresponding author: uwe.borghoff@unibw.de
A Baseline Method for Removing Invisible Image Watermarks using Deep Image Prior
Liang, Hengyue, Li, Taihui, Sun, Ju
Image watermarks have been considered a promising technique to help detect AI-generated content, which can be used to protect copyright or prevent fake image abuse. In this work, we present a black-box method for removing invisible image watermarks, without the need of any dataset of watermarked images or any knowledge about the watermark system. Our approach is simple to implement: given a single watermarked image, we regress it by deep image prior (DIP). We show that from the intermediate steps of DIP one can reliably find an evasion image that can remove invisible watermarks while preserving high image quality. Due to its unique working mechanism and practical effectiveness, we advocate including DIP as a baseline invasion method for benchmarking the robustness of watermarking systems. Finally, by showing the limited ability of DIP and other existing black-box methods in evading training-based visible watermarks, we discuss the positive implications on the practical use of training-based visible watermarks to prevent misinformation abuse.
Highly Dynamic and Flexible Spatio-Temporal Spectrum Management with AI-Driven O-RAN: A Multi-Granularity Marketplace Framework
Rasti, Mehdi, Ataeebojd, Elaheh, Taskooh, Shiva Kazemi, Monemi, Mehdi, Razmi, Siavash, Latva-aho, Matti
Current spectrum-sharing frameworks struggle with adaptability, often being either static or insufficiently dynamic. They primarily emphasize temporal sharing while overlooking spatial and spectral dimensions. We propose an adaptive, AI-driven spectrum-sharing framework within the O-RAN architecture, integrating discriminative and generative AI (GenAI) to forecast spectrum needs across multiple timescales and spatial granularities. A marketplace model, managed by an authorized spectrum broker, enables operators to trade spectrum dynamically, balancing static assignments with real-time trading. GenAI enhances traffic prediction, spectrum estimation, and allocation, optimizing utilization while reducing costs. This modular, flexible approach fosters operator collaboration, maximizing efficiency and revenue. A key research challenge is refining allocation granularity and spatio-temporal dynamics beyond existing models.
Utah bill would require cops to disclose AI-authored police reports
A bill headed to Utah's Senate floor would require police to include disclaimers in any report written with help from artificial intelligence. Introduced by Sen. Stephanie Pitcher, SB180 comes nearly a year after multiple police agencies across the country began testing software like Axon's Draft One, prompting concerns from critics and privacy advocates. Draft One was announced by Axon in April 2024, kicking off a major new phase for the company best known for manufacturing tasers and a popular line of body cameras used by law enforcement. Axon built Draft One using Microsoft's Azure OpenAI platform, and is designed to auto-generate police reports using only an officer's body cam audio records. Once processed, Draft One then crafts "a draft narrative quickly," reportedly cutting down on police officer's paperwork by as much as an hour per day.
Why AI resorts to stereotypes when it is role-playing humans
Artificial intelligence models from OpenAI and Meta often resort to simplistic and sometimes racist stereotypes when prompted to portray people of certain demographic identities – a notable flaw at a time when some tech companies and academic researchers want to replace humans with AI chatbots for some tasks. Companies such as Meta have already tried boosting engagement on social media platforms like Facebook and Instagram by deploying AI chatbots that mimic human profiles and respond to people's posts.