Large Language Model
A new Alexa AI is coming: What it will cost and when you can try it
If you've had it up to here with dumb responses from Alexa, get ready for a big change. At a glitzy event in New York City on Wednesday, Amazon unveiledโagainโits AI-enhanced and "completely re-architectured" Alexa, and we'll all soon have a chance to kick the tires ourselves. Starting next month, Amazon will kick off a public preview for the new Alexa, which it's calling Alexa . The public preview will begin gradually, with more users gradually being added to the group. This news is part of TechHive's in-depth coverage of the best smart speakers.
Nvidia to report first earnings since shock debut of Chinese AI DeepSeek
Nvidia will deliver its earnings report for the fourth quarter of 2024 Wednesday evening, which investors will be watching closely for any signs of slowing demand for semiconductor chips. The chipmaker's financials will face scrutiny over possible signals of an end to the AI-fueled market boom that has propelled the company to a stratospheric valuation of 3.1tn. Analysts expect Nvidia to maintain its leadership position as the maker of the AI industry's favorite chips, but recent news has presented new potential challenges to the company's ownership of the market. For one, analysts at TD Cowen published findings earlier in the week that Microsoft, one of Nvidia's biggest customers, was cancelling leases with private data center operators. Investors expressed concerns about the sustainability of mass investment into AI infrastructure, including Microsoft's 80bn, which would mean less spending on Nvidia's wares.
Fox News AI Newsletter: Meet the AI real estate agent making millions
Kevin O'Leary joined The Brian Kilmeade Show to discuss working with Frank McCourt to buy TikTok and the dangers of DeepSeek. ALWAYS CLOSING: Artificial intelligence is taking the world by storm and the real estate industry is no exception. Israeli startup eSelf AI is making it possible for customers to get their questions answered whether it's 3:00 in the afternoon or 3:00 in the morning. ESSENTIAL AI: NEAR A.I. co-founder and CEO Illia Polosukhin says A.I. is starting to become a fundamental part of peoples' digital life on'The Claman Countdown.' VOICE RECOGNITION CONTROVERSY: When one says "racist" into an iPhone, the voice-to-text feature indeed initially typed "Trump" before quickly correcting it to "racist." MOVING TO A FAKE CITY: There is a futuristic city designed and built from the ground up in Japan to test the latest technologies.
DeepSeek reopens AI model access as China rivalry heats up
DeepSeek has reopened access to its core programming interface after nearly a three-week suspension, resuming a service key to wider adoption of an AI model that's proven remarkably popular since its emergence last month. The 20-month-old Chinese startup, which stunned Silicon Valley and markets in January with an AI platform that rivals OpenAI's, said it's again allowing customers to top up credits for use on its application programming interface. DeepSeek suspended top-ups in early February because of capacity shortages. While those have now resumed, server resources will remain constrained during the daytime, a DeepSeek representative said in a verified company group chat on WeChat. DeepSeek resumed top-ups the same day that Alibaba Group Holding Ltd. launched a preview of its latest model, QwQ-Max, underscoring the deepening competition within China's nascent AI industry.
Talking like Piping and Instrumentation Diagrams (P&IDs)
Alimin, Achmad Anggawirya, Goldstein, Dominik P., Balhorn, Lukas Schulze, Schweidtmann, Artur M.
We propose a methodology that allows communication with Piping and Instrumentation Diagrams (P&IDs) using natural language. In particular, we represent P&IDs through the DEXPI data model as labeled property graphs and integrate them with Large Language Models (LLMs). The approach consists of three main parts: 1) P&IDs are cast into a graph representation from the DEXPI format using our pyDEXPI Python package. 2) A tool for generating P&ID knowledge graphs from pyDEXPI. 3) Integration of the P&ID knowledge graph to LLMs using graph-based retrieval augmented generation (graph-RAG). This approach allows users to communicate with P&IDs using natural language. It extends LLM's ability to retrieve contextual data from P&IDs and mitigate hallucinations. Leveraging the LLM's large corpus, the model is also able to interpret process information in PIDs, which could help engineers in their daily tasks. In the future, this work will also open up opportunities in the context of other generative Artificial Intelligence (genAI) solutions on P&IDs, and AI-assisted HAZOP studies.
Foundation Inference Models for Stochastic Differential Equations: A Transformer-based Approach for Zero-shot Function Estimation
Seifner, Patrick, Cvejoski, Kostadin, Berghaus, David, Ojeda, Cesar, Sanchez, Ramses J.
Stochastic differential equations (SDEs) describe dynamical systems where deterministic flows, governed by a drift function, are superimposed with random fluctuations dictated by a diffusion function. The accurate estimation (or discovery) of these functions from data is a central problem in machine learning, with wide application across natural and social sciences alike. Yet current solutions are brittle, and typically rely on symbolic regression or Bayesian non-parametrics. In this work, we introduce FIM-SDE (Foundation Inference Model for SDEs), a transformer-based recognition model capable of performing accurate zero-shot estimation of the drift and diffusion functions of SDEs, from noisy and sparse observations on empirical processes of different dimensionalities. Leveraging concepts from amortized inference and neural operators, we train FIM-SDE in a supervised fashion, to map a large set of noisy and discretely observed SDE paths to their corresponding drift and diffusion functions. We demonstrate that one and the same (pretrained) FIM-SDE achieves robust zero-shot function estimation (i.e. without any parameter fine-tuning) across a wide range of synthetic and real-world processes, from canonical SDE systems (e.g. double-well dynamics or weakly perturbed Hopf bifurcations) to human motion recordings and oil price and wind speed fluctuations.
Agentic Mixture-of-Workflows for Multi-Modal Chemical Search
Callahan, Tiffany J., Park, Nathaniel H., Capponi, Sara
The vast and complex materials design space demands innovative strategies to integrate multidisciplinary scientific knowledge and optimize materials discovery. While large language models (LLMs) have demonstrated promising reasoning and automation capabilities across various domains, their application in materials science remains limited due to a lack of benchmarking standards and practical implementation frameworks. To address these challenges, we introduce Mixture-of-Workflows for Self-Corrective Retrieval-Augmented Generation (CRAG-MoW) - a novel paradigm that orchestrates multiple agentic workflows employing distinct CRAG strategies using open-source LLMs. Unlike prior approaches, CRAG-MoW synthesizes diverse outputs through an orchestration agent, enabling direct evaluation of multiple LLMs across the same problem domain. We benchmark CRAG-MoWs across small molecules, polymers, and chemical reactions, as well as multi-modal nuclear magnetic resonance (NMR) spectral retrieval. Our results demonstrate that CRAG-MoWs achieve performance comparable to GPT-4o while being preferred more frequently in comparative evaluations, highlighting the advantage of structured retrieval and multi-agent synthesis. By revealing performance variations across data types, CRAG-MoW provides a scalable, interpretable, and benchmark-driven approach to optimizing AI architectures for materials discovery. These insights are pivotal in addressing fundamental gaps in benchmarking LLMs and autonomous AI agents for scientific applications.
Towards Label-Only Membership Inference Attack against Pre-trained Large Language Models
He, Yu, Li, Boheng, Liu, Liu, Ba, Zhongjie, Dong, Wei, Li, Yiming, Qin, Zhan, Ren, Kui, Chen, Chun
Membership Inference Attacks (MIAs) aim to predict whether a data sample belongs to the model's training set or not. Although prior research has extensively explored MIAs in Large Language Models (LLMs), they typically require accessing to complete output logits (\ie, \textit{logits-based attacks}), which are usually not available in practice. In this paper, we study the vulnerability of pre-trained LLMs to MIAs in the \textit{label-only setting}, where the adversary can only access generated tokens (text). We first reveal that existing label-only MIAs have minor effects in attacking pre-trained LLMs, although they are highly effective in inferring fine-tuning datasets used for personalized LLMs. We find that their failure stems from two main reasons, including better generalization and overly coarse perturbation. Specifically, due to the extensive pre-training corpora and exposing each sample only a few times, LLMs exhibit minimal robustness differences between members and non-members. This makes token-level perturbations too coarse to capture such differences. To alleviate these problems, we propose \textbf{PETAL}: a label-only membership inference attack based on \textbf{PE}r-\textbf{T}oken sem\textbf{A}ntic simi\textbf{L}arity. Specifically, PETAL leverages token-level semantic similarity to approximate output probabilities and subsequently calculate the perplexity. It finally exposes membership based on the common assumption that members are `better' memorized and have smaller perplexity. We conduct extensive experiments on the WikiMIA benchmark and the more challenging MIMIR benchmark. Empirically, our PETAL performs better than the extensions of existing label-only attacks against personalized LLMs and even on par with other advanced logit-based attacks across all metrics on five prevalent open-source LLMs.
Sliding Window Attention Training for Efficient Large Language Models
Fu, Zichuan, Song, Wentao, Wang, Yejing, Wu, Xian, Zheng, Yefeng, Zhang, Yingying, Xu, Derong, Wei, Xuetao, Xu, Tong, Zhao, Xiangyu
Recent advances in transformer-based Large Language Models (LLMs) have demonstrated remarkable capabilities across various tasks. However, their quadratic computational complexity concerning sequence length remains a significant bottleneck for processing long documents. As a result, many efforts like sparse attention and state space models have been proposed to improve the efficiency of LLMs over long sequences. Though effective, these approaches compromise the performance or introduce structural complexity. This calls for a simple yet efficient model that preserves the fundamental Transformer architecture. To this end, we introduce SWAT, which enables efficient long-context handling via Sliding Window Attention Training. This paper first attributes the inefficiency of Transformers to the attention sink phenomenon resulting from the high variance of softmax operation. Then, we replace softmax with the sigmoid function and utilize a balanced ALiBi and Rotary Position Embedding for efficient information compression and retention. Experiments demonstrate that SWAT achieves SOTA performance compared with state-of-the-art linear recurrent architectures on eight benchmarks. Code is available at https://anonymous.4open.science/r/SWAT-attention.
Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success
Kim, Moo Jin, Finn, Chelsea, Liang, Percy
Recent vision-language-action models (VLAs) build upon pretrained vision-language models and leverage diverse robot datasets to demonstrate strong task execution, language following ability, and semantic generalization. Despite these successes, VLAs struggle with novel robot setups and require fine-tuning to achieve good performance, yet how to most effectively fine-tune them is unclear given many possible strategies. In this work, we study key VLA adaptation design choices such as different action decoding schemes, action representations, and learning objectives for fine-tuning, using OpenVLA as our representative base model. Our empirical analysis informs an Optimized Fine-Tuning (OFT) recipe that integrates parallel decoding, action chunking, a continuous action representation, and a simple L1 regression-based learning objective to altogether improve inference efficiency, policy performance, and flexibility in the model's input-output specifications. We propose OpenVLA-OFT, an instantiation of this recipe, which sets a new state of the art on the LIBERO simulation benchmark, significantly boosting OpenVLA's average success rate across four task suites from 76.5% to 97.1% while increasing action generation throughput by 26$\times$. In real-world evaluations, our fine-tuning recipe enables OpenVLA to successfully execute dexterous, high-frequency control tasks on a bimanual ALOHA robot and outperform other VLAs ($\pi_0$ and RDT-1B) fine-tuned using their default recipes, as well as strong imitation learning policies trained from scratch (Diffusion Policy and ACT) by up to 15% (absolute) in average success rate. We release code for OFT and pretrained model checkpoints at https://openvla-oft.github.io/.