Goto

Collaborating Authors

 Generative AI


Towards Safer Chatbots: A Framework for Policy Compliance Evaluation of Custom GPTs

arXiv.org Artificial Intelligence

Large Language Models (LLMs) have gained unprecedented prominence, achieving widespread adoption across diverse domains and integrating deeply into society. The capability to fine-tune general-purpose LLMs, such as Generative Pre-trained Transformers (GPT), for specific tasks has facilitated the emergence of numerous Custom GPTs. These tailored models are increasingly made available through dedicated marketplaces, such as OpenAI's GPT Store. However, their black-box nature introduces significant safety and compliance risks. In this work, we present a scalable framework for the automated evaluation of Custom GPTs against OpenAI's usage policies, which define the permissible behaviors of these systems. Our framework integrates three core components: (1) automated discovery and data collection of models from the GPT store, (2) a red-teaming prompt generator tailored to specific policy categories and the characteristics of each target GPT, and (3) an LLM-as-a-judge technique to analyze each prompt-response pair for potential policy violations. We validate our framework with a manually annotated ground truth, and evaluate it through a large-scale study with 782 Custom GPTs across three categories: Romantic, Cybersecurity, and Academic GPTs. Our manual annotation process achieved an F1 score of 0.975 in identifying policy violations, confirming the reliability of the framework's assessments. The results reveal that 58.7% of the analyzed models exhibit indications of non-compliance, exposing weaknesses in the GPT store's review and approval processes. Furthermore, our findings indicate that a model's popularity does not correlate with compliance, and non-compliance issues largely stem from behaviors inherited from base models rather than user-driven customizations. We believe this approach is extendable to other chatbot platforms and policy domains, improving LLM-based systems safety.


Dance recalibration for dance coherency with recurrent convolution block

arXiv.org Artificial Intelligence

With the recent advancements in generative AI such as GAN, Diffusion, and VAE, the use of generative AI for dance generation has seen significant progress and received considerable interest. In this study, We propose R-Lodge, an enhanced version of Lodge. R-Lodge incorporates Recurrent Sequential Representation Learning named Dance Recalibration to original coarse-to-fine long dance generation model. R-Lodge utilizes Dance Recalibration method using $N$ Dance Recalibration Block to address the lack of consistency in the coarse dance representation of the Lodge model. By utilizing this method, each generated dance motion incorporates a bit of information from the previous dance motions. We evaluate R-Lodge on FineDance dataset and the results show that R-Lodge enhances the consistency of the whole generated dance motions.


Single-neuron deep generative model uncovers underlying physics of neuronal activity in Ca imaging data

arXiv.org Artificial Intelligence

Calcium imaging has become a powerful alternative to electrophysiology for studying neuronal activity, offering spatial resolution and the ability to measure large populations of neurons in a minimally invasive manner. This technique has broad applications in neuroscience, neuroengineering, and medicine, enabling researchers to explore the relationship between neuron location and activity. Recent advancements in deep generative models (DGMs) have facilitated the modeling of neuronal population dynamics, uncovering latent representations that provide insights into behavior prediction and neuronal variance. However, these models often rely on spike inference algorithms and primarily focus on population-level dynamics, limiting their applicability for single-neuron analyses. To address this gap, we propose a novel framework for single-neuron representation learning using autoregressive variational autoencoders (AVAEs). Our approach embeds individual neurons' spatiotemporal signals into a reduced-dimensional space without the need for spike inference algorithms. The AVAE excels over traditional linear methods by generating more informative and discriminative latent representations, improving tasks such as visualization, clustering, and the understanding of neuronal activity. Additionally, the reconstruction performance of the AVAE outperforms the state of the art, demonstrating its ability to accurately recover the original fluorescence signal from the learned representation. Using realistic simulations, we show that our model captures underlying physical properties and connectivity patterns, enabling it to distinguish between different firing and connectivity types. These findings position the AVAE as a versatile and powerful tool for advancing single-neuron analysis and lays the groundwork for future integration of multimodal single-cell datasets in neuroscience.


Standardizing Intelligence: Aligning Generative AI for Regulatory and Operational Compliance

arXiv.org Artificial Intelligence

Technical standards, or simply standards, are established documented guidelines and rules that facilitate the interoperability, quality, and accuracy of systems and processes. In recent years, we have witnessed an emerging paradigm shift where the adoption of generative AI (GenAI) models has increased tremendously, spreading implementation interests across standard-driven industries, including engineering, legal, healthcare, and education. In this paper, we assess the criticality levels of different standards across domains and sectors and complement them by grading the current compliance capabilities of state-of-the-art GenAI models. To support the discussion, we outline possible challenges and opportunities with integrating GenAI for standard compliance tasks while also providing actionable recommendations for entities involved with developing and using standards. Overall, we argue that aligning GenAI with standards through computational methods can help strengthen regulatory and operational compliance. We anticipate this area of research will play a central role in the management, oversight, and trustworthiness of larger, more powerful GenAI-based systems in the near future.


OpenAI announces surprise 'Deep Research' stream tonight

Engadget

OpenAI announced on X that it's hosting a livestream from Tokyo tonight, offering no more context beyond, "Deep Research." You can watch it on YouTube below. Just a few days ago, OpenAI released its new reasoning model, o3-mini. The company says it produces "more accurate and clearer answers, with stronger reasoning abilities" than its predecessor, and "works with search to find up-to-date answers with links to relevant web sources." CEO Sam Altman and other members of the OpenAI team held an AMA on Reddit on Friday to talk about it.


The AI business model is built on hype. That's the real reason the tech bros fear DeepSeek Kenan Malik

The Guardian

No, it was not a "Sputnik moment". The launch last month of DeepSeek R1, the Chinese generative AI or chatbot, created mayhem in the tech world, with stocks plummeting and much chatter about the US losing its supremacy in AI technology. Yet, for all the disruption, the Sputnik analogy reveals less about DeepSeek than about American neuroses. The original Sputnik moment came on 4 October 1957 when the Soviet Union shocked the world by launching Sputnik 1, the first time humanity had sent a satellite into orbit. It was, to anachronistically borrow a phrase from a later and even more momentous landmark, "one giant leap for mankind", in Neil Armstrong's historic words as he took a "small step" on to the surface of the moon.


Secure & Personalized Music-to-Video Generation via CHARCHA

arXiv.org Artificial Intelligence

Music is a deeply personal experience and our aim is to enhance this with a fullyautomated pipeline for personalized music video generation. Our work allows listeners to not just be consumers but co-creators in the music video generation process by creating personalized, consistent and context-driven visuals based on lyrics, rhythm and emotion in the music. The pipeline combines multimodal translation and generation techniques and utilizes low-rank adaptation on listeners' images to create immersive music videos that reflect both the music and the individual. To ensure the ethical use of users' identity, we also introduce CHARCHA, a facial identity verification protocol that protects people against unauthorized use of their face while at the same time collecting authorized images from users for personalizing their videos. This paper thus provides a secure and innovative framework for creating deeply personalized music videos. Figure 1: Image stills and lyrics from generated music videos for Rick Astley's "Never Gonna Give You Up," with character reference from CHARCHA. The videos use Queratogray Sketch[1], Western Animation Diffusion[2], and Realistic Vision V5.1[3] checkpoint models .


Guidance Source Matters: How Guidance from AI, Expert, or a Group of Analysts Impacts Visual Data Preparation and Analysis

arXiv.org Artificial Intelligence

The progress in generative AI has fueled AI-powered tools like co-pilots and assistants to provision better guidance, particularly during data analysis. However, research on guidance has not yet examined the perceived efficacy of the source from which guidance is offered and the impact of this source on the user's perception and usage of guidance. We ask whether users perceive all guidance sources as equal, with particular interest in three sources: (i) AI, (ii) human expert, and (iii) a group of human analysts. As a benchmark, we consider a fourth source, (iv) unattributed guidance, where guidance is provided without attribution to any source, enabling isolation of and comparison with the effects of source-specific guidance. We design a five-condition between-subjects study, with one condition for each of the four guidance sources and an additional (v) no-guidance condition, which serves as a baseline to evaluate the influence of any kind of guidance. We situate our study in a custom data preparation and analysis tool wherein we task users to select relevant attributes from an unfamiliar dataset to inform a business report. Depending on the assigned condition, users can request guidance, which the system then provides in the form of attribute suggestions. To ensure internal validity, we control for the quality of guidance across source-conditions. Through several metrics of usage and perception, we statistically test five preregistered hypotheses and report on additional analysis. We find that the source of guidance matters to users, but not in a manner that matches received wisdom. For instance, users utilize guidance differently at various stages of analysis, including expressing varying levels of regret, despite receiving guidance of similar quality. Notably, users in the AI condition reported both higher post-task benefit and regret.


ChartCitor: Multi-Agent Framework for Fine-Grained Chart Visual Attribution

arXiv.org Artificial Intelligence

Large Language Models (LLMs) can perform chart question-answering tasks but often generate unverified hallucinated responses. Existing answer attribution methods struggle to ground responses in source charts due to limited visual-semantic context, complex visual-text alignment requirements, and difficulties in bounding box prediction across complex layouts. We present ChartCitor, a multi-agent framework that provides fine-grained bounding box citations by identifying supporting evidence within chart images. The system orchestrates LLM agents to perform chart-to-table extraction, answer reformulation, table augmentation, evidence retrieval through pre-filtering and re-ranking, and table-to-chart mapping. ChartCitor outperforms existing baselines across different chart types. Qualitative user studies show that ChartCitor helps increase user trust in Generative AI by providing enhanced explainability for LLM-assisted chart QA and enables professionals to be more productive.


AI is not just powerful. What's really worrying is that DeepSeek has made it cheap, too John Naughton

The Guardian

Nothing cheers up a tech columnist more than the sight of 600bn being wiped off the market cap of an overvalued tech giant in a single day. And yet last Monday that's what happened to Nvidia, the leading maker of electronic picks and shovels for the AI gold rush. It was the biggest one-day slump for any company in history, and it was not alone – shares of companies in semiconductor, power and infrastructure industries exposed to AI collectively shed more than 1tn in value on the same day. The proximate cause of this chaos was the news that a Chinese tech startup of whom few had hitherto heard had released DeepSeek R1, a powerful AI assistant that was much cheaper to train and operate than the dominant models of the US tech giants – and yet was comparable in competence to OpenAI's o1 "reasoning" model. Just to illustrate the difference: R1 was said to have cost only 5.58m to build, which is small change compared with the billions that OpenAI and co have spent on their models; and R1 is about 15 times more efficient (in terms of resource use) than anything comparable made by Meta. The DeepSeek app immediately zoomed to the top of the Apple app store, where it attracted huge numbers of users who were clearly unfazed by the fact that the terms and conditions and the privacy policy they needed to accept were in Chinese.