system behavior
From Accuracy to Impact: The Impact-Driven AI Framework (IDAIF) for Aligning Engineering Architecture with Theory of Change
This paper introduces the Impact-Driven AI Framework (IDAIF), a novel architectural methodology that integrates Theory of Change (ToC) principles with modern artificial intelligence system design. As AI systems increasingly influence high-stakes domains including healthcare, finance, and public policy, the alignment problem--ensuring AI behavior corresponds with human values and intentions--has become critical. Current approaches predominantly optimize technical performance metrics while neglecting the sociotechnical dimensions of AI deployment. IDAIF addresses this gap by establishing a systematic mapping between ToC's five-stage model (Inputs-Activities-Outputs-Outcomes-Impact) and corresponding AI architectural layers (Data Layer-Pipeline Layer-Inference Layer-Agentic Layer-Normative Layer). Each layer incorporates rigorous theoretical foundations: multi-objective Pareto optimization for value alignment, hierarchical multi-agent orchestration for outcome achievement, causal directed acyclic graphs (DAGs) for hallucination mitigation, and adversarial debiasing with Reinforcement Learning from Human Feedback (RLHF) for fairness assurance. We provide formal mathematical formulations for each component and introduce an Assurance Layer that manages assumption failures through guardian architectures. Three case studies demonstrate IDAIF application across healthcare, cybersecurity, and software engineering domains. This framework represents a paradigm shift from model-centric to impact-centric AI development, providing engineers with concrete architectural patterns for building ethical, trustworthy, and socially beneficial AI systems.
Exploiting ftrace's function_graph Tracer Features for Machine Learning: A Case Study on Encryption Detection
Begovic, Kenan, Al-Ali, Abdulaziz, Malluhi, Qutaibah
This paper proposes using the Linux kernel ftrace framework, particularly the function graph tracer, to generate informative system level data for machine learning (ML) applications. Experiments on a real world encryption detection task demonstrate the efficacy of the proposed features across several learning algorithms. The learner faces the problem of detecting encryption activities across a large dataset of files, using function call traces and graph based features. Empirical results highlight an outstanding accuracy of 99.28 on the task at hand, underscoring the efficacy of features derived from the function graph tracer. The results were further validated in an additional experiment targeting a multilabel classification problem, in which running programs were identified from trace data. This work provides comprehensive methodologies for preprocessing raw trace data and extracting graph based features, offering significant advancements in applying ML to system behavior analysis, program identification, and anomaly detection. By bridging the gap between system tracing and ML, this paper paves the way for innovative solutions in performance monitoring and security analytics.
Proactive Agentic Whiteboards: Enhancing Diagrammatic Learning
Ellawela, Suveen, Gamage, Sashenka, Dissanayake, Dinithi
Educators frequently rely on diagrams to explain complex concepts during lectures, yet creating clear and complete visual representations in real time while simultaneously speaking can be cognitively demanding. Incomplete or unclear diagrams may hinder student comprehension, as learners must mentally reconstruct missing information while following the verbal explanation. Inspired by advances in code completion tools, we introduce DrawDash, an AI-powered white-board assistant that proactively completes and refines educational diagrams through multimodal understanding. Draw-Dash adopts a T AB-completion interaction model: it listens to spoken explanations, detects intent, and dynamically suggests refinements that can be accepted with a single keystroke. We demonstrate DrawDash across four diverse teaching scenarios--spanning topics from computer science and web development to biology. This work represents an early exploration into reducing instructors' cognitive load and improving diagram-based pedagogy through real-time, speech-driven visual assistance, and concludes with a discussion of current limitations and directions for formal classroom evaluation.
Rule-Based Moral Principles for Explaining Uncertainty in Natural Language Generation
Abstract--Rule-Based Moral Principles for Explaining Uncertainty in Natural Language Generation As large language models (LLMs) are increasingly used in high-stakes applications, the challenge of explaining uncertainty in natural language generation has become both a technical and moral imperative. Traditional approaches rely on probabilistic methods that are often opaque, difficult to interpret, and misaligned with human expectations of transparency and accountability. In response to these limitations, this paper introduces a novel framework based on rule-based moral principles--simple, human-inspired ethical guidelines--for responding to uncertainty in LLM-generated text. Drawing on insights from experimental moral psychology and virtue ethics, we define a set of symbolic behavioral rules such as precaution, deference, and responsibility to guide system responses under conditions of epistemic or aleatoric uncertainty. These rules are implemented declaratively and are designed to generate adaptive, context-sensitive explanations even in the absence of precise confidence metrics. The moral principles are encoded as symbolic rules within a lightweight Prolog-based engine, where each uncertainty tag (low, medium, high) activates an ethically aligned system action along with an automatically generated, plain-language rationale. We evaluate the framework through scenario-based simulations that benchmark rule coverage, assess fairness implications, and analyze trust calibration. An interpretive explanation module is integrated to reveal both the assigned uncertainty level and its underlying justification in a transparent and accessible way. We illustrate the framework through hypothetical yet plausible use cases in clinical and legal domains, demonstrating how rule-based moral reasoning can enhance user trust, promote fairness, and improve the interpretability of AI-generated language. By offering a lightweight, philosophically grounded alternative to probabilistic uncertainty modeling, our approach paves the way for more ethical, human-aligned, and socially responsible natural language generation.
Disentangling Slow and Fast Temporal Dynamics in Degradation Inference with Hierarchical Differential Models
Disentangling Slow and Fast Temporal Dynamics in Degradation Inference with Hierarchical Differential Models Mengjie Zhao, Olga Fink Learned latent states align well with true physical degradation. The framework shows robust generalization to unseen conditions. The primary latent component serves as an interpretable health indicator. Abstract Reliable inference of system degradation from sensor data is fundamental to condition monitoring and prognostics in engineered systems. Since degradation is rarely observable and measurable, it must be inferred to enable accurate health assessment and decision-making. This is particularly challenging because operational and environmental variations dominate system behavior, while degradation introduces only subtle, long-term changes. Consequently, sensor data primarily reflect short-term operational variability, making it difficult to disentangle the underlying degradation process. Residual-based methods are widely employed, but the residuals remain entangled with operational history, often resulting in noisy and unreliable degradation estimation, particularly in systems with dynamic responses. Other approaches often focus on modeling degradation-aware degradation dynamics but overlook how operational history drives long-term degradation. Neural Ordinary Equations (NODEs) offer a promising framework for inferring latent dynamics, but the time-scale separation in slow-fast systems introduces numerical stiffness and complicates training, while degradation disentanglement remains difficult. To address these limitations, we propose a novel Hierarchical Controlled Differential Equation (H-CDE) framework that incorporates a slow (degradation) and a fast (operation) CDE component in a unified architecture. Through comprehensive evaluations on both dynamic response (e.g., bridges) and steady state (e.g., aero-engine) systems, we demonstrate that H-CDE effectively disentangles degradation from operational dynamics and outperforms residual-based baselines, yielding more accurate, robust, and interpretable inference. Introduction Ensuring the reliability and safety of complex engineered systems, ranging from critical infrastructure [1] to industrial machinery [2] and aerospace structures [3], relies on continuous monitoring of their health state. A key indicator of system health is the level of degradation, whose progression enables estimation of the remaining useful life (RUL). Accurate RUL predictions support predictive maintenance strategies and help avoid both unexpected failures and unnecessarily conservative component replacements [4, 5].
Evaluating Large Language Models for Real-World Engineering Tasks
Heesch, Rene, Eilermann, Sebastian, Windmann, Alexander, Diedrich, Alexander, Rosenthal, Philipp, Niggemann, Oliver
Large Language Models (LLMs) are transformative not only for daily activities but also for engineering tasks. However, current evaluations of LLMs in engineering exhibit two critical shortcomings: (i) the reliance on simplified use cases, often adapted from examination materials where correctness is easily verifiable, and (ii) the use of ad hoc scenarios that insufficiently capture critical engineering competencies. Consequently, the assessment of LLMs on complex, real-world engineering problems remains largely unexplored. This paper addresses this gap by introducing a curated database comprising over 100 questions derived from authentic, production-oriented engineering scenarios, systematically designed to cover core competencies such as product design, prognosis, and diagnosis. Using this dataset, we evaluate four state-of-the-art LLMs, including both cloud-based and locally hosted instances, to systematically investigate their performance on complex engineering tasks. Our results show that LLMs demonstrate strengths in basic temporal and structural reasoning but struggle significantly with abstract reasoning, formal modeling, and context-sensitive engineering logic.
Taxonomizing Representational Harms using Speech Act Theory
Corvi, Emily, Washington, Hannah, Reed, Stefanie, Atalla, Chad, Chouldechova, Alexandra, Dow, P. Alex, Garcia-Gathright, Jean, Pangakis, Nicholas, Sheng, Emily, Vann, Dan, Vogel, Matthew, Wallach, Hanna
Representational harms are widely recognized among fairness-related harms caused by generative language systems. However, their definitions are commonly under-specified. We present a framework, grounded in speech act theory (Austin, 1962), that conceptualizes representational harms caused by generative language systems as the perlocutionary effects (i.e., real-world impacts) of particular types of illocutionary acts (i.e., system behaviors). Building on this argument and drawing on relevant literature from linguistic anthropology and sociolinguistics, we provide new definitions stereotyping, demeaning, and erasure. We then use our framework to develop a granular taxonomy of illocutionary acts that cause representational harms, going beyond the high-level taxonomies presented in previous work. We also discuss the ways that our framework and taxonomy can support the development of valid measurement instruments. Finally, we demonstrate the utility of our framework and taxonomy via a case study that engages with recent conceptual debates about what constitutes a representational harm and how such harms should be measured.
Dehumanizing Machines: Mitigating Anthropomorphic Behaviors in Text Generation Systems
Cheng, Myra, Blodgett, Su Lin, DeVrio, Alicia, Egede, Lisa, Olteanu, Alexandra
As text generation systems' outputs are increasingly anthropomorphic -- perceived as human-like -- scholars have also raised increasing concerns about how such outputs can lead to harmful outcomes, such as users over-relying or developing emotional dependence on these systems. How to intervene on such system outputs to mitigate anthropomorphic behaviors and their attendant harmful outcomes, however, remains understudied. With this work, we aim to provide empirical and theoretical grounding for developing such interventions. To do so, we compile an inventory of interventions grounded both in prior literature and a crowdsourced study where participants edited system outputs to make them less human-like. Drawing on this inventory, we also develop a conceptual framework to help characterize the landscape of possible interventions, articulate distinctions between different types of interventions, and provide a theoretical basis for evaluating the effectiveness of different interventions.
"I Am the One and Only, Your Cyber BFF": Understanding the Impact of GenAI Requires Understanding the Impact of Anthropomorphic AI
Cheng, Myra, DeVrio, Alicia, Egede, Lisa, Blodgett, Su Lin, Olteanu, Alexandra
Many state-of-the-art generative AI (GenAI) systems are increasingly prone to anthropomorphic behaviors, i.e., to generating outputs that are perceived to be human-like. While this has led to scholars increasingly raising concerns about possible negative impacts such anthropomorphic AI systems can give rise to, anthropomorphism in AI development, deployment, and use remains vastly overlooked, understudied, and underspecified. In this perspective, we argue that we cannot thoroughly map the social impacts of generative AI without mapping the social impacts of anthropomorphic AI, and outline a call to action.
RAPID: Robust APT Detection and Investigation Using Context-Aware Deep Learning
Amaru, Yonatan, Wudali, Prasanna, Elovici, Yuval, Shabtai, Asaf
Advanced persistent threats (APTs) pose significant challenges for organizations, leading to data breaches, financial losses, and reputational damage. Existing provenance-based approaches for APT detection often struggle with high false positive rates, a lack of interpretability, and an inability to adapt to evolving system behavior. We introduce RAPID, a novel deep learning-based method for robust APT detection and investigation, leveraging context-aware anomaly detection and alert tracing. By utilizing self-supervised sequence learning and iteratively learned embeddings, our approach effectively adapts to dynamic system behavior. The use of provenance tracing both enriches the alerts and enhances the detection capabilities of our approach. Our extensive evaluation demonstrates RAPID's effectiveness and computational efficiency in real-world scenarios. In addition, RAPID achieves higher precision and recall than state-of-the-art methods, significantly reducing false positives. RAPID integrates contextual information and facilitates a smooth transition from detection to investigation, providing security teams with detailed insights to efficiently address APT threats.