Generative AI
$K^4$: Online Log Anomaly Detection Via Unsupervised Typicality Learning
Chen, Weicong, Singh, Vikash, Rahmani, Zahra, Ganguly, Debargha, Hariri, Mohsen, Chaudhary, Vipin
--Log anomaly detection (LogAD) is crucial for identifying failures and threats in large-scale computing and cyberin-frastructure systems. However, most existing LogAD approaches suffer from key limitations: they depend on slow and error-prone log parsing, employ tightly coupled end-to-end pipelines, often require supervision for improved detection performance, and rely on flawed single-pass evaluation protocols that fail to reflect the temporal dynamics of real-world online detection. These issues significantly hinder scalability, adaptability, and the practical deployment of solutions. These descriptors inform lightweight, modular detectors, including KDE, GMM, OCSVM, and a new adaptation of DeepSVDD, which enables efficient and accurate anomaly scoring without relying on structured formats or log representation retraining. T o support realistic deployment scenarios, we also propose a principled chunk-based evaluation protocol that mimics online log ingestion, alleviates the performance overestimation and dataset undercoverage issues of prior single-pass evaluations, and enables reproducible benchmarking across datasets with varying anomaly densities. Using this setup, we conduct over 125,000 experiments across three real-world datasets (HDFS, BGL, Thunderbird), six pre-trained embedding models, four detectors, and multiple training and log sampling configurations. Logs are essential artifacts in computing systems, recording the operational behavior of applications, kernels, and user activities. This work was supported in part by the NSF research grant #2137603, #2112606, #2117439, and #2320952. These authors contributed equally to this work. With the recent surge in language models and generative AI, a growing body of work [4]-[9] has begun leveraging AI techniques to capture semantic patterns in log sequences, aiming to enable more effective LogAD.
AgentMesh: A Cooperative Multi-Agent Generative AI Framework for Software Development Automation
Software development is a complex, multi-phase process traditionally requiring collaboration among individuals with diverse expertise. We propose AgentMesh, a Python-based framework that uses multiple cooperating LLM-powered agents to automate software development tasks. In AgentMesh, specialized agents - a Planner, Coder, Debugger, and Reviewer - work in concert to transform a high-level requirement into fully realized code. The Planner agent first decomposes user requests into concrete subtasks; the Coder agent implements each subtask in code; the Debugger agent tests and fixes the code; and the Reviewer agent validates the final output for correctness and quality. We describe the architecture and design of these agents and their communication, and provide implementation details including prompt strategies and workflow orchestration. A case study illustrates AgentMesh handling a non-trivial development request via sequential task planning, code generation, iterative debugging, and final code review. We discuss how dividing responsibilities among cooperative agents leverages the strengths of large language models while mitigating single-agent limitations. Finally, we examine current limitations - such as error propagation and context scaling - and outline future work toward more robust, scalable multi-agent AI systems for software engineering automation.
Zero-shot Performance of Generative AI in Brazilian Portuguese Medical Exam
Truyts, Cesar Augusto Madid, Rabelo, Amanda Gomes, de Souza, Gabriel Mesquita, Lages, Daniel Scaldaferri, Pereira, Adriano Jose, Flato, Uri Adrian Prync, Reis, Eduardo Pontes dos, Vieira, Joaquim Edson, Silveira, Paulo Sergio Panse, Junior, Edson Amaro
Artificial intelligence (AI) has shown the potential to revolutionize healthcare by improving diagnostic accuracy, optimizing workflows, and personalizing treatment plans. Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) have achieved notable advancements in natural language processing and medical applications. However, the evaluation of these models has focused predominantly on the English language, leading to potential biases in their performance across different languages. This study investigates the capability of six LLMs (GPT-4.0 Turbo, LLaMA-3-8B, LLaMA-3-70B, Mixtral 8x7B Instruct, Titan Text G1-Express, and Command R+) and four MLLMs (Claude-3.5-Sonnet, Claude-3-Opus, Claude-3-Sonnet, and Claude-3-Haiku) to answer questions written in Brazilian spoken portuguese from the medical residency entrance exam of the Hospital das Clรญnicas da Faculdade de Medicina da Universidade de Sรฃo Paulo (HCFMUSP) - the largest health complex in South America. The performance of the models was benchmarked against human candidates, analyzing accuracy, processing time, and coherence of the generated explanations. The results show that while some models, particularly Claude-3.5-Sonnet and Claude-3-Opus, achieved accuracy levels comparable to human candidates, performance gaps persist, particularly in multimodal questions requiring image interpretation. Furthermore, the study highlights language disparities, emphasizing the need for further fine-tuning and data set augmentation for non-English medical AI applications. Our findings reinforce the importance of evaluating generative AI in various linguistic and clinical settings to ensure a fair and reliable deployment in healthcare. Future research should explore improved training methodologies, improved multimodal reasoning, and real-world clinical integration of AI-driven medical assistance.
Enhancing Materials Discovery with Valence Constrained Design in Generative Modeling
Cheng, Mouyang, Luo, Weiliang, Tang, Hao, Yu, Bowen, Cheng, Yongqiang, Xie, Weiwei, Li, Ju, Kulik, Heather J., Li, Mingda
Diffusion-based deep generative models have emerged as powerful tools for inverse materials design. Yet, many existing approaches overlook essential chemical constraints such as oxidation state balance, which can lead to chemically invalid structures. Here we introduce CrysVCD (Crystal generator with Valence-Constrained Design), a modular framework that integrates chemical rules directly into the generative process. CrysVCD first employs a transformer-based elemental language model to generate valence-balanced compositions, followed by a diffusion model to generate crystal structures. The valence constraint enables orders-of-magnitude more efficient chemical valence checking, compared to pure data-driven approaches with post-screening. When fine-tuned on stability metrics, CrysVCD achieves 85% thermodynamic stability and 68% phonon stability. Moreover, CrysVCD supports conditional generation of functional materials, enabling discovery of candidates such as high thermal conductivity semiconductors and high-$ฮบ$ dielectric compounds. Designed as a general-purpose plugin, CrysVCD can be integrated into diverse generative pipeline to promote chemical validity, offering a reliable, scientifically grounded path for materials discovery.
Large Language Model Agent for Structural Drawing Generation Using ReAct Prompt Engineering and Retrieval Augmented Generation
Zhang, Xin, Iturburu, Lissette, Villamizar, Juan Nicolas, Liu, Xiaoyu, Salmeron, Manuel, Dyke, Shirley J., Ramirez, Julio
Structural drawings are widely used in many fields, e.g., mechanical engineering, civil engineering, etc. In civil engineering, structural drawings serve as the main communication tool between architects, engineers, and builders to avoid conflicts, act as legal documentation, and provide a reference for future maintenance or evaluation needs. They are often organized using key elements such as title/subtitle blocks, scales, plan views, elevation view, sections, and detailed sections, which are annotated with standardized symbols and line types for interpretation by engineers and contractors. Despite advances in software capabilities, the task of generating a structural drawing remains labor-intensive and time-consuming for structural engineers. Here we introduce a novel generative AI-based method for generating structural drawings employing a large language model (LLM) agent. The method incorporates a retrieval-augmented generation (RAG) technique using externally-sourced facts to enhance the accuracy and reliability of the language model. This method is capable of understanding varied natural language descriptions, processing these to extract necessary information, and generating code to produce the desired structural drawing in AutoCAD. The approach developed, demonstrated and evaluated herein enables the efficient and direct conversion of a structural drawing's natural language description into an AutoCAD drawing, significantly reducing the workload compared to current working process associated with manual drawing production, facilitating the typical iterative process of engineers for expressing design ideas in a simplified way.
Unlimited Editions: Documenting Human Style in AI Art Generation
As AI art generation becomes increasingly sophisticated, HCI research has focused primarily on questions of detection, authenticity, and automation. This paper argues that such approaches fundamentally misunderstand how artistic value emerges from the concerns that drive human image production. Through examination of historical precedents, we demonstrate that artistic style is not only visual appearance but the resolution of creative struggle, as artists wrestle with influence and technical constraints to develop unique ways of seeing. Current AI systems flatten these human choices into reproducible patterns without preserving their provenance. We propose that HCI's role lies not only in perfecting visual output, but in developing means to document the origins and evolution of artistic style as it appears within generated visual traces. This reframing suggests new technical directions for HCI research in generative AI, focused on automatic documentation of stylistic lineage and creative choice rather than simple reproduction of aesthetic effects.
Assessing the Reliability of LLMs Annotations in the Context of Demographic Bias and Model Explanation
Mohammadi, Hadi, Shahedi, Tina, Mosteiro, Pablo, Poesio, Massimo, Bagheri, Ayoub, Giachanou, Anastasia
Understanding the sources of variability in annotations is crucial for developing fair NLP systems, especially for tasks like sexism detection where demographic bias is a concern. This study investigates the extent to which annotator demographic features influence labeling decisions compared to text content. Using a Generalized Linear Mixed Model, we quantify this inf luence, finding that while statistically present, demographic factors account for a minor fraction ( 8%) of the observed variance, with tweet content being the dominant factor. We then assess the reliability of Generative AI (GenAI) models as annotators, specifically evaluating if guiding them with demographic personas improves alignment with human judgments. Our results indicate that simplistic persona prompting often fails to enhance, and sometimes degrades, performance compared to baseline models. Furthermore, explainable AI (XAI) techniques reveal that model predictions rely heavily on content-specific tokens related to sexism, rather than correlates of demographic characteristics. We argue that focusing on content-driven explanations and robust annotation protocols offers a more reliable path towards fairness than potentially persona simulation.
OpenAI unleashes ChatGPT agent for truly autonomous AI tasks
OpenAI CEO Sam Altman sits down with Shannon Bream to discuss the positives and potential negatives of artificial intelligence and the importance of maintaining a lead in the A.I. industry over China. OpenAI just took a big leap forward with artificial intelligence. ChatGPT agent acts as more than just a chatbot; it serves as a real assistant that takes action on your behalf. If you've used tools like ChatGPT, Microsoft Copilot, or Google Gemini, you know they're great at answering questions and writing content. But ChatGPT agent goes beyond that.
Sam Altman just gave the best reason not to trust ChatGPT
Sam Altman, the face of ChatGPT, recently made an excellent argument for not using ChatGPT or any cloud-based AI chatbot in favor of a LLM running on your PC instead. Altman pointed out that, right now, OpenAI retains everything you tell it -- which, as Altman notes, can be everything from a casual conversation to deep, meaningful discussions about personal topics. Yes, OpenAI keeps your conversations private. But there are no legal protections requiring it to anonymize or indemnify your chats. Put another way, if a court orders OpenAI to disclose what you've told it, it probably will.
Seriously, Why Do Some AI Chatbot Subscriptions Cost More Than 200?
Why does OpenAI's monthly subscription for ChatGPT Pro cost 200? Because CEO Sam Altman said so. "I personally chose the price and thought we would make some money," Altman wrote on X. Launched late last year, the plan designed for power users includes almost unlimited access to ChatGPT as well as first dibs on feature launches, like OpenAI's new agent. The plan attracted, well, power users. A month after its initial release, Altman claimed OpenAI was still losing money on the all-you-can-eat subscription. Even though Altman admitted the 200 monthly tier was a money-loser, the release set a precedent and ushered in the vibe-based pricing era for expensive chatbot subscriptions.