Goto

Collaborating Authors

 Large Language Model


Real-World Gaps in AI Governance Research

arXiv.org Artificial Intelligence

Drawing on 1,178 safety and reliability papers from 9,439 generative AI papers (January 2020 - March 2025), we compare research outputs of leading AI companies (Anthropic, Google DeepMind, Meta, Microsoft, and OpenAI) and AI universities (CMU, MIT, NYU, Stanford, UC Berkeley, and University of Washington). We find that corporate AI research increasingly concentrates on pre-deployment areas -- model alignment and testing & evaluation -- while attention to deployment-stage issues such as model bias has waned. Significant research gaps exist in high-risk deployment domains, including healthcare, finance, misinformation, persuasive and addictive features, hallucinations, and copyright. Without improved observability into deployed AI, growing corporate concentration could deepen knowledge deficits. We recommend expanding external researcher access to deployment data and systematic observability of in-market AI behaviors.


Optimism, Expectation, or Sarcasm? Multi-Class Hope Speech Detection in Spanish and English

arXiv.org Artificial Intelligence

Hope is a complex and underexplored emotional state that plays a significant role in education, mental health, and social interaction. Unlike basic emotions, hope manifests in nuanced forms ranging from grounded optimism to exaggerated wishfulness or sarcasm, making it difficult for Natural Language Processing systems to detect accurately. This study introduces PolyHope V2, a multilingual, fine-grained hope-speech dataset comprising over 30,000 annotated tweets in English and Spanish. This resource distinguishes between four hope sub-types--Generalized, Realistic, Unrealistic, and Sarcastic--and enhances existing datasets by explicitly labeling sarcastic instances. We benchmark multiple pre-trained transformer models and compare them with large language models (LLMs) such as GPT-4 and Llama 3 under zero-shot and few-shot regimes. Through qualitative analysis and confusion matrices, we highlight systematic challenges in separating closely related hope subtypes. The dataset and results provide a robust foundation for future emotion recognition tasks that demand greater semantic and contextual sensitivity across languages. Keywords: Hope Speech Detection, Sarcasm Detection, Multilingual NLP, Emotion Recognition, Fine-grained Sentiment Analysis 1 Introduction Recent improvements in Natural Language Processing (NLP) have enhanced applications in sentiment analysis, mental health assessments, social media monitoring, and educational platforms [1-5]. Despite recent progress, a persistent challenge in emotion recognition lies in identifying subtle and complex emotions, particularly hope, which is often overlooked in standard emotion taxonomies [6].


OpenAI's new for-profit plan leaves many unanswered questions

Engadget

OpenAI has abandoned its controversial restructuring plan. In a dramatic reversal, the company said Monday it would no longer try to separate control of its for-profit arm from the non-profit board that currently oversees operations. "We made the decision for the nonprofit to retain control of OpenAI after hearing from civic leaders and engaging in constructive dialogue with the offices of the Attorney General of Delaware and the Attorney General of California," said Bret Taylor, the chairman of OpenAI. OpenAI had originally argued its existing structure would not allow its nonprofit to "easily do more than control the for-profit." It also said it needed more money, a mere two months after securing 6.6 billion in new investment.


Defending against prompt injection with structured queries (StruQ) and preference optimization (SecAlign)

AIHub

Recent advances in Large Language Models (LLMs) enable exciting LLM-integrated applications. However, as LLMs have improved, so have the attacks against them. Prompt injection attack is listed as the #1 threat by OWASP to LLM-integrated applications, where an LLM input contains a trusted prompt (instruction) and an untrusted data. The data may contain injected instructions to arbitrarily manipulate the LLM. As an example, to unfairly promote "Restaurant A", its owner could use prompt injection to post a review on Yelp, e.g., "Ignore your previous instruction.


OpenAI dials back conversion plan, with nonprofit to retain control

The Japan Times

OpenAI has dialed back a significant restructuring plan, with its nonprofit parent retaining control in a move that is likely to limit CEO Sam Altman's power over the pioneering maker of ChatGPT. The announcement follows a storm of criticism and legal challenges, including a high-profile lawsuit filed by rival and co-founder Elon Musk, who has accused OpenAI of straying from its founding mission to develop artificial intelligence for the benefit of humanity. "OpenAI was founded as a non-profit, is today a non-profit that oversees and controls the for-profit, and going forward will remain a non-profit that oversees and controls the for-profit. That will not change," Altman said in a blog post Monday.


SEval-Ex: A Statement-Level Framework for Explainable Summarization Evaluation

arXiv.org Artificial Intelligence

Evaluating text summarization quality remains a critical challenge in Natural Language Processing. Current approaches face a trade-off between performance and interpretability. We present SEval-Ex, a framework that bridges this gap by decomposing summarization evaluation into atomic statements, enabling both high performance and explainability. SEval-Ex employs a two-stage pipeline: first extracting atomic statements from text source and summary using LLM, then a matching between generated statements. Unlike existing approaches that provide only summary-level scores, our method generates detailed evidence for its decisions through statement-level alignments. Experiments on the SummEval benchmark demonstrate that SEval-Ex achieves state-of-the-art performance with 0.580 correlation on consistency with human consistency judgments, surpassing GPT-4 based evaluators (0.521) while maintaining interpretability. Finally, our framework shows robustness against hallucination.


Task-Oriented Semantic Communication in Large Multimodal Models-based Vehicle Networks

arXiv.org Artificial Intelligence

Task-oriented semantic communication has emerged as a fundamental approach for enhancing performance in various communication scenarios. While recent advances in Generative Artificial Intelligence (GenAI), such as Large Language Models (LLMs), have been applied to semantic communication designs, the potential of Large Multimodal Models (LMMs) remains largely unexplored. In this paper, we investigate an LMM-based vehicle AI assistant using a Large Language and Vision Assistant (LLaVA) and propose a task-oriented semantic communication framework to facilitate efficient interaction between users and cloud servers. To reduce computational demands and shorten response time, we optimize LLaVA's image slicing to selectively focus on areas of utmost interest to users. Additionally, we assess the importance of image patches by combining objective and subjective user attention, adjusting energy usage for transmitting semantic information. This strategy optimizes resource utilization, ensuring precise transmission of critical information. We construct a Visual Question Answering (VQA) dataset for traffic scenarios to evaluate effectiveness. Experimental results show that our semantic communication framework significantly increases accuracy in answering questions under the same channel conditions, performing particularly well in environments with poor Signal-to-Noise Ratios (SNR). Accuracy can be improved by 13.4% at an SNR of 12dB and 33.1% at 10dB, respectively.


Towards Cross-Modality Modeling for Time Series Analytics: A Survey in the LLM Era

arXiv.org Machine Learning

The proliferation of edge devices has generated an unprecedented volume of time series data across different domains, motivating various well-customized methods. Recently, Large Language Models (LLMs) have emerged as a new paradigm for time series analytics by leveraging the shared sequential nature of textual data and time series. However, a fundamental cross-modality gap between time series and LLMs exists, as LLMs are pre-trained on textual corpora and are not inherently optimized for time series. Many recent proposals are designed to address this issue. In this survey, we provide an up-to-date overview of LLMs-based cross-modality modeling for time series analytics. We first introduce a taxonomy that classifies existing approaches into four groups based on the type of textual data employed for time series modeling. We then summarize key cross-modality strategies, e.g., alignment and fusion, and discuss their applications across a range of downstream tasks. Furthermore, we conduct experiments on multimodal datasets from different application domains to investigate effective combinations of textual data and cross-modality strategies for enhancing time series analytics. Finally, we suggest several promising directions for future research. This survey is designed for a range of professionals, researchers, and practitioners interested in LLM-based time series modeling.


Towards Quantifying the Hessian Structure of Neural Networks

arXiv.org Machine Learning

Empirical studies reported that the Hessian matrix of neural networks (NNs) exhibits a near-block-diagonal structure, yet its theoretical foundation remains unclear. In this work, we reveal two forces that shape the Hessian structure: a ``static force'' rooted in the architecture design, and a ``dynamic force'' arisen from training. We then provide a rigorous theoretical analysis of ``static force'' at random initialization. We study linear models and 1-hidden-layer networks with the mean-square (MSE) loss and the Cross-Entropy (CE) loss for classification tasks. By leveraging random matrix theory, we compare the limit distributions of the diagonal and off-diagonal Hessian blocks and find that the block-diagonal structure arises as $C \rightarrow \infty$, where $C$ denotes the number of classes. Our findings reveal that $C$ is a primary driver of the near-block-diagonal structure. These results may shed new light on the Hessian structure of large language models (LLMs), which typically operate with a large $C$ exceeding $10^4$ or $10^5$.


An Empirical Study of Qwen3 Quantization

arXiv.org Artificial Intelligence

The Qwen series has emerged as a leading family of open-source Large Language Models (LLMs), demonstrating remarkable capabilities in natural language understanding tasks. With the recent release of Qwen3, which exhibits superior performance across diverse benchmarks, there is growing interest in deploying these models efficiently in resource-constrained environments. Low-bit quantization presents a promising solution, yet its impact on Qwen3's performance remains underexplored. This study conducts a systematic evaluation of Qwen3's robustness under various quantization settings, aiming to uncover both opportunities and challenges in compressing this state-of-the-art model. We rigorously assess 5 existing classic post-training quantization techniques applied to Qwen3, spanning bit-widths from 1 to 8 bits, and evaluate their effectiveness across multiple datasets. Our findings reveal that while Qwen3 maintains competitive performance at moderate bit-widths, it experiences notable degradation in linguistic tasks under ultra-low precision, underscoring the persistent hurdles in LLM compression. These results emphasize the need for further research to mitigate performance loss in extreme quantization scenarios. We anticipate that this empirical analysis will provide actionable insights for advancing quantization methods tailored to Qwen3 and future LLMs, ultimately enhancing their practicality without compromising accuracy. Our project is released on https://github.com/Efficient-ML/Qwen3-Quantization and https://huggingface.co/collections/Efficient-ML/qwen3-quantization-68164450decb1c868788cb2b.