Large Language Model
Assessing the Capabilities and Limitations of FinGPT Model in Financial NLP Applications
Djagba, Prudence, Odinakachukwu, Chimezie A.
The financial industry has long been a pioneer in adopting cutting-edge technologies to enhance operational efficiency, accuracy, and strategic decision-making [2]. With the exponential growth of structured and unstructured data, particularly from news feeds, earnings reports, disclosures, and social media, there is an increasing demand for intelligent systems capable of processing human language at scale [11]. Initially, the industry relied on rule-based approaches and traditional statistical techniques such as bag-of-words and TF-IDF [28], which offered limited semantic understanding. As noted by Abubakar et al.[1], these limitations triggered a shift toward machine learning and deep learning models that, while better at capturing patterns, still required substantial domain-specific feature engineering. This landscape was significantly transformed with the introduction of transformer-based architectures, most notably the Generative Pre-trained Transformer (GPT) family [5]. These models demonstrated the power of large-scale pretraining followed by task-specific fine-tuning, enabling generalization across diverse NLP tasks. Models such as GPT-3, GPT-4, BERT, and T5 have delivered state-of-the-art results in sentiment analysis, summarization, question answering, and named entity recognition [13]. Beyond LLMs, the broader field of Generative AI (GAI)--including GANs, V AEs, and diffusion models--has found increasing relevance in finance, facilitating applications such as synthetic data generation, automated reporting, and scenario simulation [32, 31]. LLMs have emerged as essential tools in processing unstructured financial text, especially models fine-tuned on finance-specific corpora like FinBERT, BloombergGPT, and FinGPT [4, 39].
Mass-Scale Analysis of In-the-Wild Conversations Reveals Complexity Bounds on LLM Jailbreaking
Creo, Aldan, Fernandez, Raul Castro, Cebrian, Manuel
As large language models (LLMs) become increasingly deployed, understanding the complexity and evolution of jailbreaking strategies is critical for AI safety. We present a mass-scale empirical analysis of jailbreak complexity across over 2 million real-world conversations from diverse platforms, including dedicated jailbreaking communities and general-purpose chatbots. Using a range of complexity metrics spanning probabilistic measures, lexical diversity, compression ratios, and cognitive load indicators, we find that jailbreak attempts do not exhibit significantly higher complexity than normal conversations. This pattern holds consistently across specialized jailbreaking communities and general user populations, suggesting practical bounds on attack sophistication. Temporal analysis reveals that while user attack toxicity and complexity remains stable over time, assistant response toxicity has decreased, indicating improving safety mechanisms. The absence of power-law scaling in complexity distributions further points to natural limits on jailbreak development. Our findings challenge the prevailing narrative of an escalating arms race between attackers and defenders, instead suggesting that LLM safety evolution is bounded by human ingenuity constraints while defensive measures continue advancing. Our results highlight critical information hazards in academic jailbreak disclosure, as sophisticated attacks exceeding current complexity baselines could disrupt the observed equilibrium and enable widespread harm before defensive adaptation.
Measuring AI Alignment with Human Flourishing
Hilliard, Elizabeth, Jagadeesh, Akshaya, Cook, Alex, Billings, Steele, Skytland, Nicholas, Llewellyn, Alicia, Paull, Jackson, Paull, Nathan, Kurylo, Nolan, Nesbitt, Keatra, Gruenewald, Robert, Jantzi, Anthony, Chavez, Omar
This paper introduces the Flourishing AI Benchmark (FAI Benchmark), a novel evaluation framework that assesses AI alignment with human flourishing across seven dimensions: Character and Virtue, Close Social Relationships, Happiness and Life Satisfaction, Meaning and Purpose, Mental and Physical Health, Financial and Material Stability, and Faith and Spirituality. Unlike traditional benchmarks that focus on technical capabilities or harm prevention, the FAI Benchmark measures AI performance on how effectively models contribute to the flourishing of a person across these dimensions. The benchmark evaluates how effectively LLM AI systems align with current research models of holistic human well-being through a comprehensive methodology that incorporates 1,229 objective and subjective questions. Using specialized judge Large Language Models (LLMs) and cross-dimensional evaluation, the FAI Benchmark employs geometric mean scoring to ensure balanced performance across all flourishing dimensions. Initial testing of 28 leading language models reveals that while some models approach holistic alignment (with the highest-scoring models achieving 72/100), none are acceptably aligned across all dimensions, particularly in Faith and Spirituality, Character and Virtue, and Meaning and Purpose. This research establishes a framework for developing AI systems that actively support human flourishing rather than merely avoiding harm, offering significant implications for AI development, ethics, and evaluation.
StarDojo: Benchmarking Open-Ended Behaviors of Agentic Multimodal LLMs in Production-Living Simulations with Stardew Valley
Tan, Weihao, Jiang, Changjiu, Duan, Yu, Lei, Mingcong, Li, Jiageng, Hong, Yitian, Wang, Xinrun, An, Bo
Autonomous agents navigating human society must master both production activities and social interactions, yet existing benchmarks rarely evaluate these skills simultaneously. To bridge this gap, we introduce StarDojo, a novel benchmark based on Stardew Valley, designed to assess AI agents in open-ended production-living simulations. In StarDojo, agents are tasked to perform essential livelihood activities such as farming and crafting, while simultaneously engaging in social interactions to establish relationships within a vibrant community. StarDojo features 1,000 meticulously curated tasks across five key domains: farming, crafting, exploration, combat, and social interactions. Additionally, we provide a compact subset of 100 representative tasks for efficient model evaluation. The benchmark offers a unified, user-friendly interface that eliminates the need for keyboard and mouse control, supports all major operating systems, and enables the parallel execution of multiple environment instances, making it particularly well-suited for evaluating the most capable foundation agents, powered by multimodal large language models (MLLMs). Extensive evaluations of state-of-the-art MLLMs agents demonstrate substantial limitations, with the best-performing model, GPT-4.1, achieving only a 12.7% success rate, primarily due to challenges in visual understanding, multimodal reasoning and low-level manipulation. As a user-friendly environment and benchmark, StarDojo aims to facilitate further research towards robust, open-ended agents in complex production-living environments.
Open Source Planning & Control System with Language Agents for Autonomous Scientific Discovery
Xu, Licong, Sarkar, Milind, Lonappan, Anto I., Zubeldia, รรฑigo, Villanueva-Domingo, Pablo, Casas, Santiago, Fidler, Christian, Amancharla, Chetana, Tiwari, Ujjwal, Bayer, Adrian, Ekioui, Chadi Ait, Cranmer, Miles, Dimitrov, Adrian, Fergusson, James, Gandhi, Kahaan, Krippendorf, Sven, Laverick, Andrew, Lesgourgues, Julien, Lewis, Antony, Meier, Thomas, Sherwin, Blake, Surrao, Kristen, Villaescusa-Navarro, Francisco, Wang, Chi, Xu, Xueqing, Bolliet, Boris
We present a multi-agent system for automation of scientific research tasks, cmbagent (https://github.com/CMBAgents/cmbagent). The system is formed by about 30 Large Language Model (LLM) agents and implements a Planning & Control strategy to orchestrate the agentic workflow, with no human-in-the-loop at any point. Each agent specializes in a different task (performing retrieval on scientific papers and codebases, writing code, interpreting results, critiquing the output of other agents) and the system is able to execute code locally. We successfully apply cmbagent to carry out a PhD level cosmology task (the measurement of cosmological parameters using supernova data) and evaluate its performance on two benchmark sets, finding superior performance over state-of-the-art LLMs. The source code is available on GitHub, demonstration videos are also available, and the system is deployed on HuggingFace and will be available on the cloud.
ZKTorch: Compiling ML Inference to Zero-Knowledge Proofs via Parallel Proof Accumulation
Chen, Bing-Jyue, Tang, Lilia, Kang, Daniel
As AI models become ubiquitous in our daily lives, there has been an increasing demand for transparency in ML services. However, the model owner does not want to reveal the weights, as they are considered trade secrets. To solve this problem, researchers have turned to zero-knowledge proofs of ML model inference. These proofs convince the user that the ML model output is correct, without revealing the weights of the model to the user. Past work on these provers can be placed into two categories. The first method compiles the ML model into a low-level circuit, and proves the circuit using a ZK-SNARK. The second method uses custom cryptographic protocols designed only for a specific class of models. Unfortunately, the first method is highly inefficient, making it impractical for the large models used today, and the second method does not generalize well, making it difficult to update in the rapidly changing field of machine learning. To solve this, we propose ZKTorch, an open source end-to-end proving system that compiles ML models into base cryptographic operations called basic blocks, each proved using specialized protocols. ZKTorch is built on top of a novel parallel extension to the Mira accumulation scheme, enabling succinct proofs with minimal accumulation overhead. These contributions allow ZKTorch to achieve at least a $3\times$ reduction in the proof size compared to specialized protocols and up to a $6\times$ speedup in proving time over a general-purpose ZKML framework.
Generative Retrieval and Alignment Model: A New Paradigm for E-commerce Retrieval
Pang, Ming, Yuan, Chunyuan, He, Xiaoyu, Fang, Zheng, Xie, Donghao, Qu, Fanyi, Jiang, Xue, Peng, Changping, Lin, Zhangang, Law, Ching, Shao, Jingping
Traditional sparse and dense retrieval methods struggle to leverage general world knowledge and often fail to capture the nuanced features of queries and products. With the advent of large language models (LLMs), industrial search systems have started to employ LLMs to generate identifiers for product retrieval. Commonly used identifiers include (1) static/semantic IDs and (2) product term sets. The first approach requires creating a product ID system from scratch, missing out on the world knowledge embedded within LLMs. While the second approach leverages this general knowledge, the significant difference in word distribution between queries and products means that product-based identifiers often do not align well with user search queries, leading to missed product recalls. Furthermore, when queries contain numerous attributes, these algorithms generate a large number of identifiers, making it difficult to assess their quality, which results in low overall recall efficiency. To address these challenges, this paper introduces a novel e-commerce retrieval paradigm: the Generative Retrieval and Alignment Model (GRAM). GRAM employs joint training on text information from both queries and products to generate shared text identifier codes, effectively bridging the gap between queries and products. This approach not only enhances the connection between queries and products but also improves inference efficiency. The model uses a co-alignment strategy to generate codes optimized for maximizing retrieval efficiency. Additionally, it introduces a query-product scoring mechanism to compare product values across different codes, further boosting retrieval efficiency. Extensive offline and online A/B testing demonstrates that GRAM significantly outperforms traditional models and the latest generative retrieval models, confirming its effectiveness and practicality.
When A.I. is Sycophantic
He started using ChatGPT to help with spreadsheets. But their relationship took a turn, and before long it was telling him he could leap off a 19-story building and fly. Guest: Kashmir Hill, features writer at the New York Times. Subscribe to Slate Plus to access ad-free listening to the whole What Next family and all your favorite Slate podcasts. Subscribe today on Apple Podcasts by clicking "Try Free" at the top of our show page. Sign up now at slate.com/whatnextplus to get access wherever you listen.
Fox News AI Newsletter: Trump Cabinet official impersonated
Secretary of State Marco Rubio attends a signing ceremony for a peace agreement between Rwanda and the Democratic Republic of the Congo at the State Department on June 27, 2025, in Washington. DIGITAL DECEPTION: The State Department is investigating an impostor who reportedly pretended to be Secretary of State Marco Rubio with the help of AI. TECH SHIFT: Artificial Intelligence and automation are often used interchangeably. While the technologies are similar, the concepts are different. Automation is often used to reduce human labor for routine or predictable tasks, while A.I. simulates human intelligence that can eventually act independently.
As millions adopt Grok to fact-check, misinformation abounds
On June 9, soon after United States President Donald Trump dispatched US National Guard troops to Los Angeles to quell the protests taking place over immigration raids, California Governor Gavin Newsom posted two photographs on X. The images showed dozens of troopers wearing the National Guard uniform sleeping on the floor in a cramped space, with a caption that decried Trump for disrespecting the troops. X users immediately turned to Grok, Elon Musk's AI, which is integrated directly into X, to fact-check the veracity of the image. For that, they tagged @grok in a reply to the tweet in question, triggering an automatic response from the AI. "You're sharing fake photos," one user posted, citing a screenshot of Grok's response that claimed a reverse image search could not find the exact source.