Goto

Collaborating Authors

 Large Language Model


Why AI can't take over creative writing

AIHub

In 1948, the founder of information theory, Claude Shannon, proposed modelling language in terms of the probability of the next word in a sentence given the previous words. These types of probabilistic language models were largely derided, most famously by linguist Noam Chomsky: "The notion of'probability of a sentence' is an entirely useless one." In 2022, 74 years after Shannon's proposal, ChatGPT appeared, which caught the attention of the public, with some even suggesting it was a gateway to super-human intelligence. Going from Shannon's proposal to ChatGPT took so long because the amount of data and computing time used was unimaginable even a few years before. ChatGPT is a large language model (LLM) learned from a huge corpus of text from the internet.


Who will be the next Pope? AI predicts the new head of the Roman Catholic Church after Pope Francis dies

Daily Mail - Science & tech

Following the death of Pope Francis at the age of 88, the Catholic Church must now begin the lengthy process of electing his successor. Starting at least 15 days after his death, the 135 eligible cardinals will be locked away in the legendary Conclave until they have chosen the next pope. But if you just can't wait for the world's most secretive election to run its course, MailOnline has used AI to predict the result. According to OpenAI's ChatGPT, the man set to become the next head of the Roman Catholic Church is Cardinal Pietro Parolin. As the AI points out, the 70-year-old Italian priest is seen by many as the natural heir to Pope Francis' legacy and holds an edge in current betting markets. ChatGPT said: 'As Vatican Secretary of State since 2013, Parolin is viewed as the "continuity" candidate - acceptable to both reformers and traditionalists.


I Found an Entire Book That Was Written About โ€ฆ Me. It Only Got Weirder From There.

Slate

Have you ever stared in a mirror for a few hours? Try it: Watch as your nose somehow shifts placement on your face, how your eyebrows lose symmetry, how quickly you fail to recognize yourself. Facial dysmorphia would come to anyone tasked with considering their own reflection for too long. It's a similar experience when you promote a book. For the past few weeks, I've been touring Canada and the U.S. promoting my latest book, Sucker Punch.


Nvidia CEO urges LDP to build up Japan's AI infrastructure

The Japan Times

Nvidia CEO Jensen Huang urged the ruling Liberal Democratic Party on Tuesday to build out domestic artificial intelligence infrastructure that could fuel a robotics revolution, aligning with the government's goal to boost public- and private-sector funding in AI and semiconductors. Huang's exchange with the LDP's digital committee came a day after he met with Prime Minister Shigeru Ishiba and lobbied him to generate more power to fuel AI. "You must build it yourself because it's your intelligence," said Huang, who has run the U.S. semiconductor giant since 1993 and delivered the world's first DGX-1 server to OpenAI in 2016.


a1: Steep Test-time Scaling Law via Environment Augmented Generation

arXiv.org Artificial Intelligence

Large Language Models (LLMs) have made remarkable breakthroughs in reasoning, yet continue to struggle with hallucinations, logical errors, and inability to self-correct during complex multi-step tasks. Current approaches like chain-of-thought prompting offer limited reasoning capabilities that fail when precise step validation is required. We propose Environment Augmented Generation (EAG), a framework that enhances LLM reasoning through: (1) real-time environmental feedback validating each reasoning step, (2) dynamic branch exploration for investigating alternative solution paths when faced with errors, and (3) experience-based learning from successful reasoning trajectories. Unlike existing methods, EAG enables deliberate backtracking and strategic replanning through tight integration of execution feedback with branching exploration. Our a1-32B model achieves state-of-the-art performance among similar-sized models across all benchmarks, matching larger models like o1 on competition mathematics while outperforming comparable models by up to 24.4 percentage points. Analysis reveals EAG's distinctive scaling pattern: initial token investment in environment interaction yields substantial long-term performance dividends, with advantages amplifying proportionally to task complexity. EAG's theoretical framework demonstrates how environment interactivity and systematic branch exploration together establish a new paradigm for reliable machine reasoning, particularly for problems requiring precise multi-step calculation and logical verification.


How Private is Your Attention? Bridging Privacy with In-Context Learning

arXiv.org Machine Learning

In-context learning (ICL)-the ability of transformer-based models to perform new tasks from examples provided at inference time-has emerged as a hallmark of modern language models. While recent works have investigated the mechanisms underlying ICL, its feasibility under formal privacy constraints remains largely unexplored. In this paper, we propose a differentially private pretraining algorithm for linear attention heads and present the first theoretical analysis of the privacy-accuracy trade-off for ICL in linear regression. Our results characterize the fundamental tension between optimization and privacy-induced noise, formally capturing behaviors observed in private training via iterative methods. Additionally, we show that our method is robust to adversarial perturbations of training prompts, unlike standard ridge regression. All theoretical findings are supported by extensive simulations across diverse settings.


Large Language Models Enhanced Hyperbolic Space Recommender Systems

arXiv.org Artificial Intelligence

Large Language Models (LLMs) have attracted significant attention in recommender systems for their excellent world knowledge capabilities. However, existing methods that rely on Euclidean space struggle to capture the rich hierarchical information inherent in textual and semantic data, which is essential for capturing user preferences. The geometric properties of hyperbolic space offer a promising solution to address this issue. Nevertheless, integrating LLMs-based methods with hyperbolic space to effectively extract and incorporate diverse hierarchical information is non-trivial. To this end, we propose a model-agnostic framework, named HyperLLM, which extracts and integrates hierarchical information from both structural and semantic perspectives. Structurally, HyperLLM uses LLMs to generate multi-level classification tags with hierarchical parent-child relationships for each item. Then, tag-item and user-item interactions are jointly learned and aligned through contrastive learning, thereby providing the model with clear hierarchical information. Semantically, HyperLLM introduces a novel meta-optimized strategy to extract hierarchical information from semantic embeddings and bridge the gap between the semantic and collaborative spaces for seamless integration. Extensive experiments show that HyperLLM significantly outperforms recommender systems based on hyperbolic space and LLMs, achieving performance improvements of over 40%. Furthermore, HyperLLM not only improves recommender performance but also enhances training stability, highlighting the critical role of hierarchical information in recommender systems.


PLANET: A Collection of Benchmarks for Evaluating LLMs' Planning Capabilities

arXiv.org Artificial Intelligence

Planning is central to agents and agentic AI. The ability to plan, e.g., creating travel itineraries within a budget, holds immense potential in both scientific and commercial contexts. Moreover, optimal plans tend to require fewer resources compared to ad-hoc methods. To date, a comprehensive understanding of existing planning benchmarks appears to be lacking. Without it, comparing planning algorithms' performance across domains or selecting suitable algorithms for new scenarios remains challenging. In this paper, we examine a range of planning benchmarks to identify commonly used testbeds for algorithm development and highlight potential gaps. These benchmarks are categorized into embodied environments, web navigation, scheduling, games and puzzles, and everyday task automation. Our study recommends the most appropriate benchmarks for various algorithms and offers insights to guide future benchmark development.


An LMM for Efficient Video Understanding via Reinforced Compression of Video Cubes

arXiv.org Artificial Intelligence

Large Multimodal Models (LMMs) uniformly perceive video frames, creating computational inefficiency for videos with inherently varying temporal information density. This paper present \textbf{Quicksviewer}, an LMM with new perceiving paradigm that partitions a video of nonuniform density into varying cubes using Gumbel Softmax, followed by a unified resampling for each cube to achieve efficient video understanding. This simple and intuitive approach dynamically compress video online based on its temporal density, significantly reducing spatiotemporal redundancy (overall 45$\times$ compression rate), while enabling efficient training with large receptive field. We train the model from a language backbone through three progressive stages, each incorporating lengthy videos on average of 420s/1fps thanks to the perceiving efficiency. With only 0.8M total video-text samples for training, our model outperforms the direct baseline employing a fixed partitioning strategy by a maximum of 8.72 in accuracy, demonstrating the effectiveness in performance. On Video-MME, Quicksviewer achieves SOTA under modest sequence lengths using just up to 5\% of tokens per frame required by baselines. With this paradigm, scaling up the number of input frames reveals a clear power law of the model capabilities. It is also empirically verified that the segments generated by the cubing network can help for analyzing continuous events in videos.


Interpretable Locomotion Prediction in Construction Using a Memory-Driven LLM Agent With Chain-of-Thought Reasoning

arXiv.org Artificial Intelligence

Construction workers face significant risks of work-related musculoskeletal disorders (WMSDs), driven by repetitive tasks, heavy load handling, and non-neutral postures in dynamic, unpredictable environments [1, 10]. In the U.S., construction workers experience an 11% higher WMSD rate than the average across industries, with the back and shoulders most affected [10]. While exoskeletons show promise in reducing physical strain--passive designs lowering back muscle activity by 10-40% and active ones achieving up to 80% reductions across multiple regions [5]--their practical deployment remains limited by discomfort and poor alignment with human movements, particularly in construction settings [6]. Central to these limitations is the challenge of accurately recognizing user intent across varied tasks, a gap that restricts effective collaboration [3, 34]. This misalignment heightens safety risks, as powered exoskeletons may generate destructive forces if their controlled output deviates from the user's intent [34]. Addressing this locomotion intent recognition challenge is pivotal to unlocking effective exoskeleton assistance in construction, particularly for diverse, safety-critical tasks like ladder climbing and obstacle navigation. Traditional evaluation of assistive technologies like lower-limb exoskeletons has focused narrowly on routine tasks such as straight walking [27], neglecting these critical locomotion modes and requiring a shift beyond conventional control paradigms that lack flexibility for dynamic contexts. Construction tasks are highly variable, requiring workers to adapt to shifting demands, irregular workflows, and unstructured environments where movement patterns are unpredictable [10]. This variability complicates the implementation of assistive technologies, as rigid control approaches struggle to accommodate rapid task transitions and environmental uncertainty.